Using Terraform in Azure Pipelines, without extensions

Azure Pipelines offers a variety of extensions for integrating Terraform:

• An official one from Microsoft which has a number of limitations and has not been receiving any updates for years.
• A very popular one developed by my colleague Charles Zipp, which offers great functionality, but not all enterprises wish to install community extensions for Azure DevOps.
• An extension developed by HashiCorp themselves, which has also not been further developed after a first shot.

Thankfully, it is actually not necessary at all to use Azure Pipelines extensions to use Terraform effectively in Azure DevOps. These extensions are simple wrappers around the Terraform command line, and you can easily manage those steps yourself with simple script tasks. That is what I’ve been doing successfully in many customer projects.

Use an AzureCLI task with the option addSpnToEnvironment: true to obtain the details of the service connection, and pass it to Terraform.

We need to provide authentication for the state storage (typically in an Azure Storage blob), and for the resource manager in order to create resources. The Terraform Azure provider can use the variables ARM_CLIENT_ID, etc. to initialize its connection to Azure. In the sample below, we also piggyback on those variables to set the backend-config for state storage, but you could also use another service principal (and perhaps subscription) for that.

You can use this template to get started and adapt it to your setup.

parameters:
- name: TerraformStateKey
  type: string
- name: TerraformDirectory
  default: my/path/to/terraform/files
- name: TerraformVersion
  default: 0.12.24

- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
  displayName: Install Terraform
  inputs:
    terraformVersion: ${{ parameters.TerraformVersion }}

- task: AzureCLI@1
  displayName: Terraform credentials
  inputs:
    azureSubscription: My-Service-Connection
    scriptLocation: inlineScript
    inlineScript: |
      set -eu  # fail on error
      subscriptionId=$(az account show --query id -o tsv)
      echo "##vso[task.setvariable variable=ARM_CLIENT_ID]$servicePrincipalId"
      echo "##vso[task.setvariable variable=ARM_CLIENT_SECRET;issecret=true]$servicePrincipalKey"
      echo "##vso[task.setvariable variable=ARM_SUBSCRIPTION_ID]$subscriptionId"
      echo "##vso[task.setvariable variable=ARM_TENANT_ID]$tenantId"
    addSpnToEnvironment: true

- task: AzureCLI@1
  displayName: Terraform init
  inputs:
    azureSubscription: ${{ parameters.TerraformBackendServiceConnection }}
    scriptLocation: inlineScript
    inlineScript: |
      set -eux  # fail on error
      subscriptionId=$(az account show --query id -o tsv)
      terraform init \
        -backend-config=storage_account_name=${{ parameters.TerraformBackendStorageAccount }} \
        -backend-config=container_name=${{ parameters.TerraformBackendStorageContainer }} \
        -backend-config=key=${{ parameters.environment }}.tfstate \
        -backend-config=resource_group_name=${{ parameters.TerraformBackendResourceGroup }} \
        -backend-config=subscription_id="$(ARM_SUBSCRIPTION_ID)" \
        -backend-config=tenant_id="$(ARM_TENANT_ID)" \
        -backend-config=client_id="$(ARM_CLIENT_ID)" \
        -backend-config=client_secret="$(ARM_CLIENT_SECRET)"    workingDirectory: ${{ parameters.TerraformDirectory }}
    addSpnToEnvironment: true

- bash: |
    set -eu  # fail on error
    terraform plan -out=tfplan -input=false
    terraform apply -input=false -auto-approve tfplan
  displayName: Terraform apply
  workingDirectory: ${{ parameters.TerraformDirectory }}

See this other post for passing Terraform outputs to further tasks.