Automating Infrastructure Provisioning with Terraform and Azure DevOps

Infrastructure as Code (IaC) is a crucial practice for modern software development teams. It allows us to manage and provision infrastructure through code, which can be versioned, reviewed, and automated. Terraform is a popular IaC tool that enables you to define and provide data center infrastructure using a declarative configuration language. When combined with Azure DevOps, it allows for automated and continuous delivery of infrastructure changes.

In this tutorial, we'll explore how to set up a project that uses Terraform to provision infrastructure on Azure, all managed through Azure DevOps. This approach facilitates consistent and reliable infrastructure deployments, reducing manual errors and increasing efficiency.

Prerequisites

Before starting, ensure you have the following tools installed:

1. Terraform: Install Terraform using the following command:

   # For macOS
   brew tap hashicorp/tap
   brew install hashicorp/tap/terraform
   
   # For Windows
   choco install terraform
   
   # For Linux
   sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl
   curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
   sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
   sudo apt-get update && sudo apt-get install terraform
   

2. Azure CLI: Install Azure CLI with the following command:

   # For macOS
   brew install azure-cli
   
   # For Windows
   choco install azure-cli
   
   # For Linux
   curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
   

3. Azure DevOps Account: Sign up for an Azure DevOps account if you don't have one.

Project Structure

We'll start by creating a basic directory structure for our Terraform project. This will help organize our configuration files and scripts.

terraform-azure-devops/
├── main.tf
├── variables.tf
├── outputs.tf
└── .gitignore

Step 1: Initialize a Terraform Project

First, we need to initialize a new Terraform project. This involves creating a main.tf file that will define our infrastructure.

main.tf

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "East US"
}

This file does the following:

• Configures the Azure provider.
• Creates a resource group named example-resources in the "East US" region.

After creating the main.tf file, initialize the Terraform project by running:

terraform init

This command downloads the necessary provider plugins and prepares the directory for other Terraform commands.

Step 2: Define Variables

To make our configuration more flexible, we can define variables in a variables.tf file.

variables.tf

variable "resource_group_name" {
  description = "The name of the resource group"
  default     = "example-resources"
}

variable "location" {
  description = "The location of the resource group"
  default     = "East US"
}

This file defines two variables:

• resource_group_name with a default value of "example-resources".
• location with a default value of "East US".

Update the main.tf file to use these variables:

resource "azurerm_resource_group" "example" {
  name     = var.resource_group_name
  location = var.location
}

Step 3: Output Values

To verify the resources created by Terraform, we can define outputs in an outputs.tf file.

outputs.tf

output "resource_group_name" {
  description = "The name of the resource group"
  value       = azurerm_resource_group.example.name
}

output "resource_group_location" {
  description = "The location of the resource group"
  value       = azurerm_resource_group.example.location
}

This file specifies that after a successful apply, Terraform will output the name and location of the resource group.

To apply these configurations and create the resources, run the following commands:

terraform plan
terraform apply

The terraform plan command shows the changes that will be made, while terraform apply executes the plan and creates the resources in Azure. After applying, you'll see the output values defined in outputs.tf.

In the next part of this tutorial, we will continue by integrating this setup with Azure DevOps for automated deployments.

Step 4: Integrate with Azure DevOps

Now that we have our Terraform configuration set up, the next step is to automate the deployment process using Azure DevOps. This involves creating a pipeline that will apply our Terraform configurations.

Create a New Pipeline

1. Navigate to Azure DevOps: Log in to your Azure DevOps account and navigate to your project.

2. Create a New Pipeline: Go to the Pipelines section and click on "New Pipeline".

3. Select a Source: Choose the repository where your Terraform files are stored.

4. Configure Your Pipeline: Use the YAML option to define your pipeline. Here's a sample azure-pipelines.yml file:

   trigger:
     branches:
       include:
         - main
   
   pool:
     vmImage: 'ubuntu-latest'
   
   steps:
     - task: UsePythonVersion@0
       inputs:
         versionSpec: '3.x'
         addToPath: true
   
     - script: |
         pip install --upgrade pip
         pip install -r requirements.txt
       displayName: 'Install dependencies'
   
     - task: AzureCLI@2
       inputs:
         azureSubscription: '<Your Azure Subscription>'
         scriptType: 'bash'
         scriptLocation: 'inlineScript'
         inlineScript: |
           az login --service-principal -u $(clientId) -p $(clientSecret) --tenant $(tenantId)
           az account set --subscription $(subscriptionId)
   
     - script: |
         terraform init
         terraform plan
         terraform apply -auto-approve
       displayName: 'Run Terraform'
   

   Replace <Your Azure Subscription> and the service principal placeholders with your actual Azure subscription details.

5. Save and Run the Pipeline: Save the pipeline and run it to ensure everything is working correctly.

Step 5: Secure Your Pipeline

It's important to manage sensitive information securely. Azure DevOps provides a way to store secrets using variable groups.

Create a Variable Group

1. Navigate to Pipelines: In Azure DevOps, go to Pipelines > Library.

2. Create a Variable Group: Click on "Variable group" and create a new one. Add variables like clientId, clientSecret, tenantId, and subscriptionId.

3. Link Variable Group: In your pipeline YAML, link the variable group:

   variables:
   - group: 'YourVariableGroupName'
   

This ensures that sensitive information is not hardcoded in your pipeline YAML.

Complete Working Example

Here's how your final directory structure and files should look:

terraform-azure-devops/
├── main.tf
├── variables.tf
├── outputs.tf
├── .gitignore
└── azure-pipelines.yml

Complete main.tf

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "example" {
  name     = var.resource_group_name
  location = var.location
}

Complete variables.tf

variable "resource_group_name" {
  description = "The name of the resource group"
  default     = "example-resources"
}

variable "location" {
  description = "The location of the resource group"
  default     = "East US"
}

Complete outputs.tf

output "resource_group_name" {
  description = "The name of the resource group"
  value       = azurerm_resource_group.example.name
}

output "resource_group_location" {
  description = "The location of the resource group"
  value       = azurerm_resource_group.example.location
}

Complete azure-pipelines.yml

trigger:
  branches:
    include:
      - main

pool:
  vmImage: 'ubuntu-latest'

variables:
- group: 'YourVariableGroupName'

steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.x'
      addToPath: true

  - script: |
      pip install --upgrade pip
      pip install -r requirements.txt
    displayName: 'Install dependencies'

  - task: AzureCLI@2
    inputs:
      azureSubscription: '<Your Azure Subscription>'
      scriptType: 'bash'
      scriptLocation: 'inlineScript'
      inlineScript: |
        az login --service-principal -u $(clientId) -p $(clientSecret) --tenant $(tenantId)
        az account set --subscription $(subscriptionId)

  - script: |
      terraform init
      terraform plan
      terraform apply -auto-approve
    displayName: 'Run Terraform'

Common Errors and Fixes

Error: Error: Could not load plugin

Fix: Ensure that you have initialized your Terraform project with terraform init before running any other Terraform commands.

Error: Authentication failed

Fix: Verify that your Azure service principal credentials are correct and that they are stored securely in Azure DevOps variable groups.

Error: Resource already exists

Fix: Ensure that the resource names are unique or use Terraform's state management commands to handle existing resources.

Conclusion

In this tutorial, we set up a Terraform project to provision Azure infrastructure and automated the deployment using Azure DevOps. This setup allows for consistent, repeatable, and automated infrastructure management, enhancing productivity and reducing the potential for errors.

Sources

• Terraform Documentation
• Azure DevOps Documentation
• Azure CLI Documentation