Querying external data with Terraform

In the previous two recipes, we learned that it is possible to use either the data block or the terraform_remote_state block to retrieve external data. However, there are scenarios where the data block does not exist in the provider or terraform_remote_state cannot be used, such as when we need to process with an external API or need to use a local tool and process its output.

To meet this need, there is an external resource in Terraform that allows you to call an external program and retrieve its output data so that it can be used in the Terraform configuration.

Use of the external provider imposes prerequisites that may not be obvious (for example, in this case, we expect a particular version of PowerShell) or may be difficult to communicate other than through README files or documentation. Also, Terraform is generally designed to work the same cross-platform (operating system/architecture), but this essentially restricts the configuration to particular platforms that can (and do) run PowerShell – presumably just Windows. These requirements apply to both CI and local environments.

In this recipe, we will learn how to call an external program and retrieve its output so that we can reuse it.

Getting ready

For this recipe, we will use an existing Terraform configuration that allows us to provision a Resource Group in Azure.

Here, we want a Resource Group to be in a different Azure region (location), depending on the environment (dev or production).

The source code for this recipe is available at https://github.com/PacktPublishing/Terraform-Cookbook/tree/master/CHAP02/external.

How to do it…

Perform the following steps:

1. In the directory that contains our main.tf file, create a PowerShell GetLocation.ps1 script that contains the following content:

# Read the JSON payload from stdin
$jsonpayload = [Console]::In.ReadLine()

# Convert JSON to a string
$json = ConvertFrom-Json $jsonpayload
$environment = $json.environment

if($environment -eq "Production"){
 $location="westeurope"
}else{
 $location="westus"
}

# Write output to stdout
Write-Output "{ ""location"" : ""$location""}"

2. In the main.tf file, add the external block, as follows:

data "external" "getlocation" {
  program = ["Powershell.exe", "./GetLocation.ps1"]
  query = {
      environment = "${var.environment_name}"
  }
}

3. Then, modify the code of the Resource Group to make its location more dynamic, as follows:

resource "azurerm_resource_group" "rg" {
  name = "RG-${local.resource_name}"
  location = data.external.getlocation.result.location
}

4. Optionally, you can add an output value that has the following configuration:

output "locationname" {
  value = data.external.getlocation.result.location
}

How it works…

In step 1, we wrote the PowerShell GetLocation.ps1 script, which will be called by Terraform locally. This script takes in environment as an input parameter in JSON format. Then, this PowerShell script makes a condition on this input environment and returns the right Azure region as output so that we can use it in our Terraform configuration.

Then, in step 2, we used the Terraform external resource, which calls this PowerShell script and provides it with the contents of the environment_name variable as a parameter.

Finally, in step 3, we used the return value of this external block in the location property of the Resource Group.

The following screenshot shows the output of executing terraform plan with the environment_name variable, which is set to Dev:

As you can see, the regional location of the Resource Group is westus.

The following screenshot shows the output executing terraform plan with the environment_name variable, which is set to Production:

As you can see, the location of the Resource Group is westeurope.

As we saw in the Manipulating variables recipe, in this example, we used the -var option of the terraform plan command, which allows us to assign a value to a variable upon executing the command.

Optionally, we can also add a Terraform output that exposes this value. This can be displayed upon executing Terraform. This can also be exploited at other places in the Terraform configuration.

The following screenshot shows the output after running the terraform apply command:

As we can see, the terraform output command displays the right locationname value.

There's more…

In this recipe, we used a PowerShell script, but this script also works with all the other scripting languages and tools that are installed on your local machine.

This external resource contains specifics about the protocol, the format of the parameters, and its output. I advise that you read its documentation to learn more: https://www.terraform.io/docs/providers/external/data_source.html

See also

The following are some example articles regarding how to use the external Terraform resource:

• https://dzone.com/articles/lets-play-with-terraform-external-provider
• https://thegrayzone.co.uk/blog/2017/03/external-terraform-provider-powershell/