Requirements for infrastructure provisioner

Before proceeding to the existing solutions, let's lay out the list of most important requirements for a tool such as this, so we are able to choose it wisely.

Supports a wide variety of services

AWS alone already has dozens of entities to take care of. Other players (DigitalOcean, Google Cloud, Microsoft Azure, and so on) increase this number significantly. And if you want to add smaller SaaS providers to the game, you get hundreds of resources to manage.

Idempotency

Same as with a single server configuration, reapplying infrastructure template should not do the job twice. If you have a template defining 50 different resources, from EC2 instances to S3 buckets, then you do not want to duplicate or recreate all of them every time you apply the template. You want only missing parts to be created, existing ones to be in the desired state and the ones, which have become obsolete to be destroyed.

Dependency resolution

It is important to be able not just to define 2 app servers, 1 DB server and 2 security groups,but to also point them to each other using some lookup mechanism. Especially when creating a complete environment from scratch, you want to ensure the correct order of creation to achieve a flawless bootstrap of each component.

Note

Here and further in the book, the term environment will mean a complete set of resources that infrastructure consists of. It includes a network setup, all servers, and all related resources.

Robust integration with existing tools

Even though it is pretty awesome to have all infrastructure in one beautiful template, you still need to take care of what is happening on each particular server: applications need to be deployed, databases need to be configured, and so on. This is not the job for infrastructure provisioning tool. But certainly a tool like this should easily integrate with other tools such as Chef, which solves this problem already.

Platform agnosticism

Ideally, templates should be platform agnostic. This means that if I define a template for 2 app servers, 1 db server, all talk to each other, I should be able to easily switch from AWS to local Vagrant without rewriting the template. Platform agnosticism is difficult to obtain, while at the same time, might not really be needed that often. Completely changing the underlying platform is a rather rare event that happens perhaps once or twice in a product's lifetime.

Smart update management

This is a tricky one, and at the moment of writing, no tool can do it flawlessly in every case (and, honestly, unlikely it will ever do). What happens when I change a type of three EC2 instances from m3.medium to c4.xlarge? Will my m3.medium instances shut down and be replaced one by one with new ones? Will they be instantly destroyed leading to a few minutes of downtime? Or will the tool just ignore the updated instance type? Or will it not and then just override old nodes and I will end up with three new nodes and three old EC2 instances that I have to remove manually? Solutions to this problem differ from platform to platform, which makes it more complicated for the tool to be platform agnostic.

Ease of extension

The last requirement is of particular importance: there must be an easy way to extend this tool to support other resources. For example, if a tool lacks support for AWS Kinesis or particular feature or property of already supported service, and there is no plan to support it officially, then there has to be a way to implement it yourself quickly.