Historically, infrastructure deployment has undergone three key technological changes:

• The datacenter period
• The virtualization period
• The cloud period

In the datacenter period, infrastructure deployment and configuration were fully manual. Right from putting up the racks; stacking the servers, routers, and switches; installing operating systems and cabling them; and ensuring offsite backup for disaster recovery. There was very little scope for automation in infrastructure tasks. Automation tasks were limited to installing the OS via the Unattended Install option, configuring the software, and so on.

In the virtualization period, infrastructure deployment got better. VMware was the pioneer in creating production-grade virtualization offerings. Some of the time-consuming tasks, such as operating system installation and application configuration, could then be deployed in a matter of minutes. Further, centralized backup and patching helped reduce the duration and complexity of tasks. Although virtual machines could be provisioned instantly, this was limited by the hardware you had already purchased for your VMware-based setup in your datacenter.

Some automation started taking place at this stage with HashiCorp’s Vagrant tool.

Vagrant helped quickly provision the development environment by adding the required resources and their dependencies in the Vagrant configuration files. These files were then used by the Vagrant tool to create the virtual machine in the VMware Workstation. This helped with quickly creating test environments and pre-production environments to test certain features before moving them to production. Since the provisioning and de-provisioning of the servers was done using the configuration files, the process could be automated easily, making it quicker and less error-prone.

Though the amount of infrastructure automation increased at this stage, it was limited to vendor-specific tools. Professionals were still using VMware-specific tools and technologies to automate production-grade setups. The same tools could not be carried to other vendors’ solutions.

To summarize the discussion so far, the following were the challenges of infrastructure automation in the datacenter and virtualization periods:

• Manual processes were error-prone
• High expenses
• A long time was needed to set up the infrastructure required for application deployment
• A lack of tight integrations between various products
• Dependence on vendor-specific automation tools in the virtualization world
• Deploying applications in a different geographical region was a herculean task

It was the cloud era that paved the way for infrastructure automation in a significant manner by solving some of the above key challenges.

Why Is the Cloud Model a Good Fit for Infrastructure Automation?

The cloud model is considered suitable for infrastructure automation for the following reasons:

• Any creation/deletion/modification of a resource in the cloud can be done by calling an API
• There is seamless integration between the various resources of the cloud that can be stitched together for a production-grade solution
• You can access the same set of resources in different geographical regions to deploy the same solution
• The compute, network, security, and SaaS tools are software-defined and can be created, modified, and integrated into the virtual environment

Now that you have reviewed the history of infrastructure provisioning and the challenges associated with each period, you are ready to explore the automation of infrastructure provisioning using IaC.

As the name implies, IaC refers to managing infrastructure resources in the form of code instead of manual provisioning. This involves the creation, modification, and deletion of all the infrastructure resources via code.

One of the key principles of DevOps is automation. IaC precisely fits into this principle. Apart from automation, IaC also provides the advantage of using the same best practices that are used for application code in the Software Development Life Cycle (SDLC). This implies that, now, even infrastructure could be versioned and pipelines can be created for continuous deployment since it is all in code.

The following section presents the key advantages that IaC provides over manual deployments.

Advantages of IaC

While looking at the advantages of IaC, it will be compared against manual deployment. Some of the key advantages are listed below:

• Quicker deployment timelines: When the whole solution is in code that has been vetted, tested, and approved, it is easy to deploy everything in one go. This takes less time than the same deployment being done manually.
• Consistently repeatable deployments: As the resources are provisioned with the same set of APIs, you can expect the same behavior every single time it gets deployed. The uniformity of the deployment behavior provides assurance about the stability of the solution.
• Version controlled via a source code management tool such as Git: The configurations and integrations of all the resources can be carried out in code, which is stored in a tool such as Git. This helps you with versioning, merge requests, approvals, and so on. If there is an issue with the latest deployment, it is easy to roll back to the previous version.
• Better operational efficiency: Automating the process of launching and managing resources allows quicker deployment, which frees up the operations team to work on other important items rather than spending time just setting up the environments.
• Self-service: If you want to empower the development team to deploy the infrastructure, you can set certain guidelines for the modules/templates. They can use these guidelines for self-service and don’t have to depend on other teams.
• Accountability: All code written, every modification made, and every line deleted is tracked in the version control system. It is easy to assign accountability to the person responsible for any tasks performed.
• Increased security: IaC embeds security from the base level and in each layer, such as the network, app, and database layers. Once these are validated by the infosec team, they can be used by all teams. This improves the overall security posture of the organization.

In the next section, you will review the various techniques used in the industry for provisioning infrastructure automatically and how they compare against Terraform.

Various Options for Implementing IaC

There are many ways of implementing IaC. Which option you choose depends on various factors, such as the level of automation required, the skills available in the team, the cloud platform chosen for application deployment, the plan for a multi-cloud presence, and so on. In the following sections, you will go through the options that are regularly used in the industry. Though there are options, Terraform has emerged as a go-to tool for IaC.

Ad Hoc Scripts

Ad hoc scripts are typically written in Shell script, Perl, or Python to automate some of the infrastructure provisioning by directly calling the API and writing the required logic to integrate the resource into the solution. The disadvantage of this is that there is no standardization, and hence each person may solve a problem using different logic and resources in the scripting languages. Scripts written today may not make sense to the same person after three months.

Configuration Management Tools

Configuration management tools such as Chef, Puppet, and Ansible are meant to be used for managing the configuration of software within the operating system. These tools also support infrastructure provisioning. All three of these tools were launched before Terraform and were used by engineers for infrastructure automation. However, this is not their primary functionality. It is important to use the right tool for the right job. Using the wrong tool could give sub-optimal results or could require more effort from you to achieve the same result that could have been achieved using the right tool with minimal effort.

If you want to create the infrastructure for a three-tier architecture-based solution, you may end up spending a similar amount of time on all three tools to create the initial infrastructure. However, the complexity starts when you start modifying the infrastructure.

Consider an example where you want to increase the number of servers from three to six:

• Write commands that will give the number of servers running in the account
• Write logic to calculate the new instances to be launched
• Finally, write code to launch these additional instances

In the case of Terraform, it is as simple as changing the number of servers from three to six. Terraform takes care of figuring out what needs to be done to get the servers to six.

Cloud-Based IaC Services

Each of the major cloud vendors has its own service for IaC functionality:

• AWS has CloudFormation and Cloud Development Kit (CDK)
• Microsoft Azure has Azure Resource Manager
• GCP has Cloud Deployment Manager

Each of these services has very tight integration with the services of the particular cloud, and their support for new services in that cloud will be significantly quicker than any third-party tool, such as Terraform or Pulumi. However, if you need to be present in multiple clouds, are unsure about sticking with a single cloud provider, or just want the team to learn how to use one tool that can be used across the infrastructure, platform, and SaaS tools provisioning automation, then it is better to choose a tool like Terraform that is not dependent on any single vendor but works across them all.

Cloud-Agnostic IaC Tools

Terraform by HashiCorp is a pioneer in cloud-agnostic IaC tools (i.e., able to run on any cloud without getting tied to a single cloud). In recent years, a new tool called Pulumi has also been slowly adopted. Pulumi lets users write code to deploy applications in the language of their choice. Currently, it supports Node.js, Python, Go, .NET, Java, and YAML format.

Note

The AWS Cloud Development Kit (AWS CDK) lets you define the AWS cloud infrastructure in a general-purpose programming language such as TypeScript, JavaScript, Python, Java, C#/.NET, or Go. Both Pulumi and AWS CDK expect you to have some programming language knowledge to make the best use of the tool.

Terraform is an IaC tool that lets you create and manage your infrastructure by writing code in a simple language called HashiCorp Configuration Language (HCL).

The following section describes the features of Terraform and explains how the problems of manual provisioning and other IaC options are solved by Terraform.

Features of Terraform

We will explore the primary features of Terraform in this section.

Cloud/Vendor Agnostic

A tool that is very specifically oriented toward a particular platform becomes highly dependent on the features of the platform and hinders customers’ ability to switch to another platform when they want to. It is very important for an IaC tool to be cloud-/vendor-agnostic (i.e., able to run on any cloud without getting tied to a single cloud/vendor) for such customers. Unlike many IaC tools provided by cloud vendors, Terraform is fully cloud-/vendor-agnostic and works with all the major cloud providers and also the majority of other vendors.

A Pioneer in IaC

Terraform was launched by HashiCorp in 2014 when the IT industry used either scripts, cloud-specific tools, or configuration management tools for infrastructure automation. It pioneered a new way of solving the infra-automation problem and brought in multi-cloud support. This was done by using a declarative approach to provisioning infrastructure, along with having “current state and desired state” as the central idea, where Terraform assesses the current infrastructure state with the desired state of infrastructure as defined by the user and then makes relevant changes to change the current state to the desired state. Trust in the HashiCorp brand prompted customers to use Terraform in production even before the general availability of version 1.0, which was announced in 2021. The tool has only improved with newer releases, by adding more features, such as HCP Terraform integration, moved blocks for code refactoring, support for Open Policy Agent (OPA), the ability to import manually created resources, testing frameworks, and so on, and integration with new partners.

Wide Partner Integration

Terraform can be used to provision and manage resources on any of the cloud platforms and SaaS offerings. Terraform already has thousands of partners integrated with it. Partners typically integrate with Terraform by creating a plugin that is downloaded by the customer along with the Terraform binary. There is a new set of partners who have products for code scanning, observability, cost management, security, and so on. If you want to support the automation of your product via Terraform, you can write your own custom provider plugin.

Declarative

Procedural and declarative ways of coding are an important consideration in understanding IaC.

In the procedural style, the focus is on clearly defining the steps to achieve the desired end state. Ad hoc scripts and tools such as Ansible and Chef are all procedural language-based. In contrast, declarative style only requires you to outline the end state and the tool takes care of driving the workflow to this end. Terraform uses a declarative approach for infrastructure automation. Hence, the code is easier to write for a newbie.

Idempotent

When you run the same command/instruction multiple times and achieve the same result as you got the first time, the command/instruction is called idempotent. Terraform is idempotent. For example, if you have a Terraform file that creates an EC2 instance (a virtual machine in AWS) and you run it for the first time, it will create an EC2 instance. Running it a second or third time will not create additional instances as the desired state has already been achieved.

Easy Learning Curve

Terraform supports two formats to write and manage configuration files: JSON and HashiCorp Configuration Language (HCL).

JSON is typically used by systems for parsing but is tough for humans. In contrast, HCL is very easy to learn and implement even for someone with no programming background.

Version Controlled

The Terraform code written for infrastructure management is managed using a source code management tool such as Git. This code is pushed to platforms such as GitHub, GitLab, and so on to keep it in a central location. Storing it centrally helps with team collaborations, rolling back to previous versions in case of issues with the latest version, and creating a pipeline for automated infrastructure deployment.

Automation

Manual provisioning of infrastructure is manageable for a simple use case. When you are dealing with the creation of thousands of resources, the manual method will cause delays and errors, and will also be expensive. Automation solves all these problems.

You may be tempted to do things manually, but anything that needs to be done more than once should be considered for automation. There is a one-time investment of time while you write the code that will then bring you the benefits of automation when you have to provision the same or similar resource multiple times.

Documentation

Documentation is crucial to explain the current state of your architecture and resources, but it gets out of date quickly in the cloud world. When Terraform is used for full management of the infrastructure, the Terraform code itself can give you the latest state of the resource or the solution that is deployed. Please note that using Terraform does not take away the need for documentation but can help to reduce exhaustive documentation.

Community Support

Terraform is widely used and supported by the community. Whenever cloud vendors add new features to existing products or launch new services, the community quickly adds them to Terraform and creates a merge request with the owners of the repo. Any bugs are also quickly detected and raised with the owners of the plugin for a fix.

Terraform was an open source tool until August 10, 2023, and it changed to the community edition when HashiCorp changed the licensing of their products from Mozilla Public License v2.0 (MPL 2.0) to Business Source License (BSL or BUSL) v1.1.

Though the BSL license is open, free, and makes the source code available, it does not meet the “open source” criteria defined by The Open Source Initiative (OSI), because of which the Terraform tool cannot be called “open source” anymore. This license is applicable from Terraform version 1.5.5.

As per the new license, “Organizations providing competitive offerings to HashiCorp will no longer be permitted to use the community edition products free of charge under our BSL license.”

For more details on new licensing and how it impacts your environment, visit the following links:

• https://www.hashicorp.com/blog/hashicorp-adopts-business-source-license
• https://www.hashicorp.com/license-faq

In this chapter, you learned about the challenges of the manual provisioning of infrastructure, what IaC is, and what its advantages are. You also reviewed the importance of Terraform in the IaC landscape and examined what differentiates it from other tools on the market. This information will help you understand the big picture of infrastructure automation and the tools available on the market to solve the problems associated with it. This understanding will help you have a meaningful technical conversation with others about infrastructure automation.

There was a short discussion about the change of license that has stripped the term “open source” from the Terraform feature list.

Now that you understand why Terraform is required, you are ready to explore the basics of Terraform and its workflow in Chapter 3, Basics of Terraform and Core Workflow.

Apart from a solid understanding of key concepts, being able to think quickly under time pressure is a skill that will help you ace your certification exam. That is why working on these skills early on in your learning journey is key.

Chapter review questions are designed to improve your test-taking skills progressively with each chapter you learn and review your understanding of key concepts in the chapter at the same time. You’ll find these at the end of each chapter.

How to Access these Resources

To learn how to access these resources, head over to the chapter titled Chapter 11, Accessing the Online Practice Resources.

To open the Chapter Review Questions for this chapter, perform the following steps:

1. Click the link – https://packt.link/HCorp003Ch2.

   Alternatively, you can scan the following QR code (Figure 2.1):

Figure 2.1 – QR code that opens Chapter Review Questions for logged-in users

2. Once you log in, you’ll see a page similar to the one shown in Figure 2.2:

Figure 2.2 – Chapter Review Questions for Chapter 2

3. Once ready, start the following practice drills, re-attempting the quiz multiple times.

Exam Readiness Drill

For the first three attempts, don’t worry about the time limit.

ATTEMPT 1

The first time, aim for at least 40%. Look at the answers you got wrong and read the relevant sections in the chapter again to fix your learning gaps.

ATTEMPT 2

The second time, aim for at least 60%. Look at the answers you got wrong and read the relevant sections in the chapter again to fix any remaining learning gaps.

ATTEMPT 3

The third time, aim for at least 75%. Once you score 75% or more, you start working on your timing.

Tip

You may take more than three attempts to reach 75%. That’s okay. Just review the relevant sections in the chapter till you get there.

Target: Your aim is to keep the score the same while trying to answer these questions as quickly as possible. Here’s an example of how your next attempts should look like:

Table 2.1 – Sample timing practice drills on the online platform

Note

The time limits shown in the above table are just examples. Set your own time limits with each attempt based on the time limit of the quiz on the website.

With each new attempt, your score should stay above 75% while your “time taken” to complete should “decrease”. Repeat as many attempts as you want till you feel confident dealing with the time pressure.