Introduction to Terraform

Welcome to this introductory guide to Terraform. For anyone who is new to Terraform and unaware of what it is, as well as for the purpose of comparison with other IaC tools that are currently associated with major cloud providers including AWS, Azure, and Google, we believe that this is the best guide to begin with. In this guide, we will be focusing on what Terraform is and what problems it can solve for you, undertaking a comparison with other software tools, including ARM templates, AWS CloudFormation, and Google Cloud Deployment Manager, and explaining how you can start using Terraform effectively in your day-to-day jobs related to the provisioning and maintenance of your IT infrastructure.

What is Terraform?

Terraform is one of the open source tools that was introduced to the market by HashiCorp in 2014 as IaC software (IaC means we can write code for our infrastructure) that is mainly used for building, changing, and managing infrastructure safely and efficiently. Terraform can help with multi-cloud environments by having a single workflow, in other words, terraform init, terraform plan, terraform apply, and so on, for all clouds. The infrastructure that Terraform manages can be hosted on public clouds such as AWS, Microsoft Azure, and GCP, or on-premises in private clouds such as VMware vSphere, OpenStack, or CloudStack. Terraform handles IaC, so you never have to worry about your infrastructure drifting away from its desired configuration.

Terraform mainly uses Terraform files ending with .tf or .tf.json that hold detailed information about what infrastructure components are required in order to run a single application or your entire data center. Terraform generates an execution plan, which describes what it is going to do to reach the desired state, and then executes it to build the infrastructure described. If there is any change in the configuration file, Terraform is able to determine what has been changed and create incremental execution plans that can be applied.

Terraform can not only manage low-level components, such as compute instances, storage, and networking; it can also support high-level components, such as DNS and SaaS features, provided that the resource API is available from the providers.

After learning about what Terraform is, you might have one more question in your mind: what exactly makes this Terraform so popular? To answer that question, first and foremost, Terraform is cloud-agnostic, which means you can provision or manage your infrastructure in any cloud platform. The second thing that makes Terraform very much in demand is its standard workflow. You don't need to remember N number of parts of the workflow; a simple init, plan, and apply from Terraform's point of view would be enough and it is the same across any platform. The third factor is the Terraform syntaxing. Terraform uses uniform code syntaxing whether you work on any cloud or on-premises. There are many more exceptional factors that could encourage enterprise customers to start using Terraform.

Features of Terraform

Let's now try to get an understanding of all of Terraform's features, which are pushing up market demand for the product.

Infrastructure as code

Infrastructure is defined in a code format based on proper syntax in a configuration file that can be shared and reused. The code defined in the configuration file will provide a blueprint of your data center or the resource that you are planning to deploy. You should be able to deploy a complete infrastructure from that configuration file following the Terraform workflow.

Execution plans

The Terraform workflow has three steps – init, plan, and apply. During the planning step, it generates an execution plan. The execution plan gives you information about what Terraform will do when you call apply. This means you do not get any sort of surprise when you perform terraform apply.

Note

We are going to cover in detail the Terraform workflow in the upcoming chapters. So, stay tuned.

Resource graph

Terraform builds a graph of all your resources and parallelizes the creation and modification of any non-dependent resources. Because of this resource graph, Terraform manages to build infrastructure as efficiently as possible that is sufficiently intelligent to understand dependencies in its infrastructure.

Changing automation

Complex changes to your defined infrastructure can be applied with minimal human interaction. With the aforementioned execution plan and resource graph, you know exactly what Terraform will change and in what order, thereby avoiding multiple possible human errors.

Terraform use cases

As we have got to know what Terraform is, let's now learn about some of the use cases of Terraform in the enterprise world. A few of them have been discussed as follows.

Heroku app setup

Heroku is one of the most popular Platforms as a Service (PaaS) for hosting web apps. Developers use it to create an app and then attach other services, such as a database or email provider. One of the best features of the Heroku app is its ability to elastically scale the number of dynos or workers. However, most non-trivial applications quickly require many add-ons and external services:

Figure 1.1 – Heroku app architecture

By using Terraform, entire things that are required for a Heroku application setup could be codified in a configuration file, thereby ensuring that all the required add-ons are available, and the best part of this is that with the help of Terraform, all of this can be achieved in just 60 seconds. Any changes requested by the developer in the Heroku app can be actioned immediately using Terraform, whether it be a complex task related to configuring DNS to set a CNAME or setting up Cloudflare as a content delivery network (CDN) for the app, and so on and so forth.

Multi-tier applications

N-tier architecture deployment is quite common across the industry when thinking about the required infrastructure for an application. Generally, two-tier architecture is more in demand. This is a pool of web servers and a database tier. As per the application requirements, additional tiers can be added for API servers, caching servers, routing meshes, and so on. This pattern is used because each tier can be scaled independently and without disturbing other tiers:

Figure 1.2 – N-tier application architecture

Now, let's try to understand how Terraform can support us in achieving N-tier application infrastructure deployment. In the Terraform configuration file, each tier can be described as a collection of resources, and the dependencies between the resources for each tier can either be implicit or we can define them explicitly so that we can easily control the sequence of the resource deployment. This helps us to manage each tier separately without disturbing the others.

Self-service clusters

In a large organization, it's quite challenging for the central operation team to provide infrastructure to the product team as and when needed. The product team should be able to create and maintain their infrastructure using tooling provided by the central operations team:

Figure 1.3 – Self-service cluster

In the preceding requirement, the entire infrastructure can be codified using Terraform, which will focus on building and scaling the infrastructure, and a Terraform configuration file can be shared within an organization, enabling product teams to use the configuration as a black box and use Terraform as a tool to manage their services. During deployment of the infrastructure, if the product team encounters any issues, they can reach out to the central operations team for help and support.

Software demos

Nowadays, software development is increasing by the day, and it is very difficult to get the infrastructure required to test that software. We have tools such as Vagrant at our disposal to help us build virtualized environments, and while you may be able to use that environment for demonstration purposes, it is really difficult to perform software demos directly on production infrastructure:

Figure 1.4 – Software demo example

A software developer can provide a Terraform configuration to create, provision, and bootstrap a demo on cloud providers such as Azure, GCP, and AWS. This allows end users to easily demo the software on their infrastructure, and it even allows them to perform scale-in or scale-out of the infrastructure.

Disposable environments

In the industry, it is quite common to have multiple landscapes, including production, staging, or development environments. These environments are generally designed as a subset of the production environment, so as and when any application needs to be deployed and tested, it can easily be done in the smaller environment; but the problem with the increase in complexity of the infrastructure is that it's very difficult to manage it:

Figure 1.5 – Multiple environments

Using Terraform, the production environment that you constructed can be written in a code format, and then it can be shared with other environments, such as staging, QA, or dev. This configuration code can be used to spin up any new environments to perform testing, and can then be easily removed when you are done testing. Terraform can help to maintain a parallel environment and it can provide an option in terms of its scalability.

Software-defined networking

Software-Defined Networking (SDN) is quite famous in data centers, as it allows operators to operate a software-defined network very smoothly and developers are able to develop their applications, which can easily be run on top of the network infrastructure provided. The control layer and infrastructure layer are the two main components for defining a software-defined network:

Figure 1.6 – Software-defined network

Software-defined networks can be transformed into code using Terraform. The configuration code written in Terraform can automatically set up and modify settings by interfacing with the control layer. This allows the configuration to be versioned and changes to be automated. As an example, Azure Virtual Network is one of the most commonly used SDN implementations and can be configured by Terraform.

Resource schedulers

In large-scale infrastructures, the static assignment of applications to machines is very challenging. In terms of Terraform, there are many schedulers available, such as Borg, Mesos, YARN, and Kubernetes, that can be used to overcome this problem. These can be used to dynamically schedule Docker containers, Hadoop, Spark, and many other software tools:

Figure 1.7 – Kubernetes with Terraform

Terraform is not just limited to cloud providers such as Azure, GCP, and AWS. Resource schedulers can also behave as providers, enabling Terraform to request resources from them. This allows Terraform to be used in layers, to set up the physical infrastructure running the schedulers, as well as provisioning them on the scheduled grid. There is a Kubernetes provider that can be configured using Terraform to schedule any Pod deployment. You can read about Kubernetes with Terraform at https://learn.hashicorp.com/collections/terraform/kubernetes.

Multi-cloud deployment

Nowadays, every organization is moving toward multi-cloud, and one of the challenging tasks is to deploy the entire infrastructure in a different cloud. Every cloud provider has its own defined manner of deployment, such as ARM templates for Azure or AWS CloudFormation. Hence, it is very difficult for an administrator to learn about all of these while maintaining the complexity of the environment deployment:

Figure 1.8 – Multi-cloud deployment

Realizing the complexity of multi-cloud infrastructure deployments using already-existing tools that are very specific to each cloud provider, HashiCorp came up with an approach known as Terraform. Terraform is cloud-agnostic. A single configuration can be used to manage multiple providers, and it can even handle cross-cloud dependencies. This simplifies management and orchestration, helping administrators to handle large-scale, multi-cloud infrastructures.

So far, we have covered IaC, namely, Terraform, its features, and the different use case scenarios where we can apply Terraform. Furthermore, we have covered how Terraform differs from other IaCs mainly used in the major cloud providers, including AWS, Azure, and Google.