Arindam Mitra

Posted on May 31, 2022 • Updated on Jun 16, 2023

Terraform plan in Devops GUI

#azure #devops #terraform #iac

Greetings my fellow Technology Advocates and Specialists.

In this Session, I will demonstrate how to Publish Terraform Plan in Azure DevOps Graphical User Interface (GUI).

LIVE RECORDED SESSION:-
LIVE DEMO was Recorded as part of my Presentation in AZURE BACK TO SCHOOL - 2022 Forum/Platform
Duration of My Demo = 37 Mins 04 Secs

THIS IS HOW IT LOOKS AT THE END!!!

REQUIREMENTS:-

Azure Subscription.
Azure DevOps Organisation and Project.
Service Principal with Delegated Graph API Rights and Required RBAC (Typically Contributor on Subscription or Resource Group)
Azure Resource Manager Service Connection in Azure DevOps.
Azure Pipelines Terraform Tasks Extension by Charles Zipp Installed in Azure DevOps.

CODE REPOSITORY:-

arindam0310018 / 31-May-2022-DevOps__Publish-Terraform-Plan-In-DevOps-GUI

TERRAFORM PLAN IN DEVOPS GUI

Greetings my fellow Technology Advocates and Specialists.

In this Session, I will demonstrate how to Publish Terraform Plan in Azure DevOps Graphical User Interface (GUI).

LIVE RECORDED SESSION:-
LIVE DEMO was Recorded as part of my Presentation in AZURE BACK TO SCHOOL - 2022 Forum/Platform
Duration of My Demo = 37 Mins 04 Secs

THIS IS HOW IT LOOKS AT THE END!!!

REQUIREMENTS:-

Azure Subscription.
Azure DevOps Organisation and Project.
Service Principal with Delegated Graph API Rights and Required RBAC (Typically Contributor on Subscription or Resource Group)
Azure Resource Manager Service Connection in Azure DevOps.
Azure Pipelines Terraform Tasks Extension by Charles Zipp Installed in Azure DevOps.

EXTENSION DETAILS:-
NAME:
Azure Pipelines Terraform Tasks
WHERE TO FIND:
https://marketplace.visualstudio.com/items?itemName=charleszipp.azure-pipelines-tasks-terraform&targetId=11c5414f-ba26-4659-87a7-aa40610cf74a&utm_source=vstsproduct&utm_medium=ExtHubManageList

INSTALLED IN AZURE DEVOPS ORGANISATION:

HOW DOES MY CODE PLACEHOLDER LOOKS LIKE:-

OBJECTIVE:-
Deploy a Resource Group and Log Analytics Workspace.
Publish the Terraform Plan in Azure

…

View on GitHub

EXTENSION DETAILS:-
NAME:
Azure Pipelines Terraform Tasks
WHERE TO FIND:
https://marketplace.visualstudio.com/items?itemName=charleszipp.azure-pipelines-tasks-terraform&targetId=11c5414f-ba26-4659-87a7-aa40610cf74a&utm_source=vstsproduct&utm_medium=ExtHubManageList

INSTALLED IN AZURE DEVOPS ORGANISATION:

HOW DOES MY CODE PLACEHOLDER LOOKS LIKE:-

OBJECTIVE:-
Deploy a Resource Group and Log Analytics Workspace.
Publish the Terraform Plan in Azure DevOps GUI.

PIPELINE CODE SNIPPET:-

AZURE DEVOPS YAML PIPELINE (azure-pipelines-Publish-TF-Plan-GUI-v1.0.yml):-



###############################
#PIPELINE TRIGGER CONDITION:-
###############################
trigger: none

######################
#DECLARE PARAMETERS:-
######################

parameters:
- name: envName
  displayName: Select Environment
  default: NonProd
  values:
  - NonProd

- name: actionToPerform
  displayName: Deploy or Destroy
  default: Deploy
  values:
  - Deploy

######################
#DECLARE VARIABLES:-
######################
variables:
  ServiceConnection: amcloud-cicd-service-connection
  resourceGroup: tfpipeline-rg
  storageAccount: tfpipelinesa
  storageAccountSku: Standard_LRS
  container: terraform
  tfstateFile: PUBLISH-TF-PLAN/LogaPublishTFPlan.tfstate
  BuildAgent: ubuntu-latest
  terraform_ver: latest
  workingDir: $(System.DefaultWorkingDirectory)/Publish-TF-Plan-In-GUI
  target: $(build.artifactstagingdirectory)/AMTF
  artifact: AM

#########################
# Declare Build Agents:-
#########################
pool:
  vmImage: $(BuildAgent)

###################
# Declare Stages:-
###################
stages:

- stage: PUBLISH_PLAN
  jobs:
  - job: PUBLISH
    displayName: PUBLISH
    steps:
# Install Terraform Installer in the Build Agent:-
    - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
      displayName: INSTALL TERRAFORM VERSION
      inputs:
        terraformVersion: '$(terraform_ver)'
# Terraform Init:-
    - task: TerraformCLI@0
      displayName: TERRAFORM INIT
      inputs:
        command: 'init'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        backendServiceArm: '$(ServiceConnection)' 
        backendAzureRmResourceGroupName: '$(resourceGroup)' 
        backendAzureRmStorageAccountName: '$(storageAccount)'
        backendAzureRmStorageAccountSku: '$(storageAccountSku)'
        backendAzureRmContainerName: '$(container)'
        backendAzureRmKey: '$(tfstateFile)'
# Terraform Validate:-
    - task: TerraformCLI@0
      displayName: TERRAFORM VALIDATE
      inputs:
        command: 'validate'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        environmentServiceName: '$(ServiceConnection)'
# Terraform Plan:-
    - task: TerraformCLI@0
      displayName: TERRAFORM PLAN
      inputs:
        command: 'plan'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        commandOptions: "--var-file=loga.tfvars --out=tfplan"
        environmentServiceName: '$(ServiceConnection)'
        publishPlanResults: 'tfplan'

- stage: BUILD
  jobs:
  - job: BUILD
    displayName: BUILD
    steps:
# Install Terraform Installer in the Build Agent:-
    - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
      displayName: INSTALL TERRAFORM VERSION
      inputs:
        terraformVersion: '$(terraform_ver)'
# Terraform Init:-
    - task: TerraformCLI@0
      displayName: TERRAFORM INIT
      inputs:
        command: 'init'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        backendServiceArm: '$(ServiceConnection)' 
        backendAzureRmResourceGroupName: '$(resourceGroup)' 
        backendAzureRmStorageAccountName: '$(storageAccount)'
        backendAzureRmStorageAccountSku: '$(storageAccountSku)'
        backendAzureRmContainerName: '$(container)'
        backendAzureRmKey: '$(tfstateFile)'
# Terraform Validate:-
    - task: TerraformCLI@0
      displayName: TERRAFORM VALIDATE
      inputs:
        command: 'validate'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        environmentServiceName: '$(ServiceConnection)'
# Terraform Plan:-
    - task: TerraformCLI@0
      displayName: TERRAFORM PLAN
      inputs:
        command: 'plan'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        commandOptions: "--var-file=loga.tfvars --out=tfplan"
        environmentServiceName: '$(ServiceConnection)'
# Copy Files to Artifacts Staging Directory:-
    - task: CopyFiles@2
      displayName: COPY FILES ARTIFACTS STAGING DIRECTORY
      inputs:
        SourceFolder: '$(workingDir)'
        Contents: |
          **/*.tf
          **/*.tfvars
          **/*tfplan*
        TargetFolder: '$(target)'
# Publish Artifacts:-
    - task: PublishBuildArtifacts@1
      displayName: PUBLISH ARTIFACTS
      inputs:
        targetPath: '$(target)'
        artifactName: '$(artifact)' 

- stage: DEPLOY
  condition: |
     and(succeeded(),
       eq('${{ parameters.actionToPerform }}', 'Deploy'), 
       eq(variables['build.sourceBranch'], 'refs/heads/main')
     )
  jobs:
  - deployment: 
    displayName: DEPLOY
    environment: '${{ parameters.envName }}'
    pool:
      vmImage: $(BuildAgent)
    strategy:
      runOnce:
        deploy:
          steps:
# Download Artifacts:-
          - task: DownloadBuildArtifacts@0
            displayName: DOWNLOAD ARTIFACTS
            inputs:
              buildType: 'current'
              downloadType: 'single'
              artifactName: '$(artifact)'
              downloadPath: '$(System.ArtifactsDirectory)' 
# Install Terraform Installer in the Build Agent:-
          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
            displayName: INSTALL TERRAFORM VERSION
            inputs:
              terraformVersion: '$(terraform_ver)'
# Terraform Init:-
          - task: TerraformCLI@0
            displayName: TERRAFORM INIT
            inputs:
              command: 'init'
              backendType: 'azurerm'
              workingDirectory: '$(System.ArtifactsDirectory)/$(artifact)/AMTF/' 
              backendServiceArm: '$(ServiceConnection)' 
              backendAzureRmResourceGroupName: '$(resourceGroup)' 
              backendAzureRmStorageAccountName: '$(storageAccount)'
              backendAzureRmStorageAccountSku: '$(storageAccountSku)'
              backendAzureRmContainerName: '$(container)'
              backendAzureRmKey: '$(tfstateFile)'
# Terraform Apply:-
          - task: TerraformCLI@0
            displayName: TERRAFORM APPLY 
            inputs:
              command: 'apply'
              backendType: 'azurerm'
              workingDirectory: '$(System.ArtifactsDirectory)/$(artifact)/AMTF'
              commandOptions: '--var-file=loga.tfvars'
              environmentServiceName: '$(ServiceConnection)'

Now, let me explain each part of YAML Pipeline for better understanding.

PART #1:-

BELOW FOLLOWS PIPELINE RUNTIME VARIABLES CODE SNIPPET:-



######################
#DECLARE PARAMETERS:-
######################

parameters:
- name: envName
  displayName: Select Environment
  default: NonProd
  values:
  - NonProd

- name: actionToPerform
  displayName: Deploy or Destroy
  default: Deploy
  values:
  - Deploy

THIS IS HOW IT LOOKS WHEN YOU EXECUTE THE PIPELINE FROM AZURE DEVOPS:-

NOTE:-
No User Input Required.

PART #2:-

BELOW FOLLOWS PIPELINE VARIABLES CODE SNIPPET:-



######################
#DECLARE VARIABLES:-
######################
variables:
  ServiceConnection: amcloud-cicd-service-connection
  resourceGroup: tfpipeline-rg
  storageAccount: tfpipelinesa
  storageAccountSku: Standard_LRS
  container: terraform
  tfstateFile: PUBLISH-TF-PLAN/LogaPublishTFPlan.tfstate
  BuildAgent: ubuntu-latest
  terraform_ver: latest
  workingDir: $(System.DefaultWorkingDirectory)/Publish-TF-Plan-In-GUI
  target: $(build.artifactstagingdirectory)/AMTF
  artifact: AM

NOTE:-
Please feel free to change the values of the variables.
The entire YAML pipeline is build using Parameters and variables. No Values are Hardcoded.
"Working Directory" Path should be based on your Code Placeholder.

PART #3:-

PIPELINE STAGE DETAILS FOLLOW BELOW:-

This is a 3 Stage Pipeline with 2 Runtime Variables - 1) DevOps Environment, and 2) Action To Perform
The Names of the Stages are - 1) PUBLISH_PLAN 2) BUILD, and 3) DEPLOY

PIPELINE STAGE - PUBLISH_PLAN:-



- stage: PUBLISH_PLAN
  jobs:
  - job: PUBLISH
    displayName: PUBLISH
    steps:
# Install Terraform Installer in the Build Agent:-
    - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
      displayName: INSTALL TERRAFORM VERSION
      inputs:
        terraformVersion: '$(terraform_ver)'
# Terraform Init:-
    - task: TerraformCLI@0
      displayName: TERRAFORM INIT
      inputs:
        command: 'init'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        backendServiceArm: '$(ServiceConnection)' 
        backendAzureRmResourceGroupName: '$(resourceGroup)' 
        backendAzureRmStorageAccountName: '$(storageAccount)'
        backendAzureRmStorageAccountSku: '$(storageAccountSku)'
        backendAzureRmContainerName: '$(container)'
        backendAzureRmKey: '$(tfstateFile)'
# Terraform Validate:-
    - task: TerraformCLI@0
      displayName: TERRAFORM VALIDATE
      inputs:
        command: 'validate'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        environmentServiceName: '$(ServiceConnection)'
# Terraform Plan:-
    - task: TerraformCLI@0
      displayName: TERRAFORM PLAN
      inputs:
        command: 'plan'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        commandOptions: "--var-file=loga.tfvars --out=tfplan"
        environmentServiceName: '$(ServiceConnection)'
        publishPlanResults: 'tfplan'

PUBLISH_PLAN STAGE PERFORMS BELOW:-

##	TASKS
1.	Terraform Installer installed in Azure DevOps Build Agent.
2.	Terraform Init
3.	Terraform Validate
4.	Terraform Plan
5.	Publish Terraform Plan in Azure DevOps GUI.

NOTE:-



- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0

EXPLANATION:-
Instead of using TerraformInstaller@0 YAML Task, I have specified the Full Name. This is because I have two Terraform Extensions in my DevOps Organisation and with each of the Terraform Extension, exists the Terraform Install Task
The Names of the Extensions are listed below:-
1. Terraform by Microsoft DevLabs
2. Azure Pipelines Terraform Tasks by Charles Zipp
If Full Name is not provided, then below Error is Encountered:-

Alternatively, below can also be used as Full Name:-



- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-installer.TerraformInstaller@0

DIFFERENCES BETWEEN TERRAFORM EXTENSIONS (Terraform by Microsoft DevLabs VS Azure Pipelines Terraform Tasks by Charles Zipp) :-

CATEGORY	TERRAFORM BY MICROSOFT DEVLABS	AZURE PIPELINES TERRAFORM TASKS BY CHARLES ZIPP
Terraform Installer Task	- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0	- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-installer.TerraformInstaller@0
Terraform Task	TerraformTaskV2@2	TerraformCLI@0
Terraform Init Input Parameter	This is not Available	backendAzureRmStorageAccountSku
Terraform Init, Validate, Plan and Apply Input Parameter	provider: 'azurerm'	backendType: 'azurerm'
Terraform Validate and Plan Input Parameter	environmentServiceNameAzureRM	environmentServiceName
Terraform Plan Input Parameter	This is not Available	publishPlanResults

HOW TERRAFORM PLAN IS GETTING PUBLISHED IN AZURE DEVOPS GUI :-
This is achieved by using the publishPlanResults Input Parameters in the Terraform Plan Task in Publish_Plan Stage



# Terraform Plan:-
    - task: TerraformCLI@0
      displayName: TERRAFORM PLAN
      inputs:
        command: 'plan'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        commandOptions: "--var-file=loga.tfvars --out=tfplan"
        environmentServiceName: '$(ServiceConnection)'
        publishPlanResults: 'tfplan'

PIPELINE STAGE - BUILD:-



- stage: BUILD
  jobs:
  - job: BUILD
    displayName: BUILD
    steps:
# Install Terraform Installer in the Build Agent:-
    - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
      displayName: INSTALL TERRAFORM VERSION
      inputs:
        terraformVersion: '$(terraform_ver)'
# Terraform Init:-
    - task: TerraformCLI@0
      displayName: TERRAFORM INIT
      inputs:
        command: 'init'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        backendServiceArm: '$(ServiceConnection)' 
        backendAzureRmResourceGroupName: '$(resourceGroup)' 
        backendAzureRmStorageAccountName: '$(storageAccount)'
        backendAzureRmStorageAccountSku: '$(storageAccountSku)'
        backendAzureRmContainerName: '$(container)'
        backendAzureRmKey: '$(tfstateFile)'
# Terraform Validate:-
    - task: TerraformCLI@0
      displayName: TERRAFORM VALIDATE
      inputs:
        command: 'validate'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        environmentServiceName: '$(ServiceConnection)'
# Terraform Plan:-
    - task: TerraformCLI@0
      displayName: TERRAFORM PLAN
      inputs:
        command: 'plan'
        backendType: 'azurerm'
        workingDirectory: '$(workingDir)'
        commandOptions: "--var-file=loga.tfvars --out=tfplan"
        environmentServiceName: '$(ServiceConnection)'
# Copy Files to Artifacts Staging Directory:-
    - task: CopyFiles@2
      displayName: COPY FILES ARTIFACTS STAGING DIRECTORY
      inputs:
        SourceFolder: '$(workingDir)'
        Contents: |
          **/*.tf
          **/*.tfvars
          **/*tfplan*
        TargetFolder: '$(target)'
# Publish Artifacts:-
    - task: PublishBuildArtifacts@1
      displayName: PUBLISH ARTIFACTS
      inputs:
        targetPath: '$(target)'
        artifactName: '$(artifact)'

BUILD STAGE PERFORMS BELOW:-

##	TASKS
1.	Terraform Installer installed in Azure DevOps Build Agent.
2.	Terraform Init
3.	Terraform Validate
4.	Terraform Plan
5.	Copy the Terraform files (Most Importantly Terraform Plan Output) to Artifacts Staging Directory.
6.	Publish Artifacts

PIPELINE STAGE - DEPLOY:-



- stage: DEPLOY
  condition: |
     and(succeeded(),
       eq('${{ parameters.actionToPerform }}', 'Deploy'), 
       eq(variables['build.sourceBranch'], 'refs/heads/main')
     )
  jobs:
  - deployment: 
    displayName: DEPLOY
    environment: '${{ parameters.envName }}'
    pool:
      vmImage: $(BuildAgent)
    strategy:
      runOnce:
        deploy:
          steps:
# Download Artifacts:-
          - task: DownloadBuildArtifacts@0
            displayName: DOWNLOAD ARTIFACTS
            inputs:
              buildType: 'current'
              downloadType: 'single'
              artifactName: '$(artifact)'
              downloadPath: '$(System.ArtifactsDirectory)' 
# Install Terraform Installer in the Build Agent:-
          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
            displayName: INSTALL TERRAFORM VERSION
            inputs:
              terraformVersion: '$(terraform_ver)'
# Terraform Init:-
          - task: TerraformCLI@0
            displayName: TERRAFORM INIT
            inputs:
              command: 'init'
              backendType: 'azurerm'
              workingDirectory: '$(System.ArtifactsDirectory)/$(artifact)/AMTF/' 
              backendServiceArm: '$(ServiceConnection)' 
              backendAzureRmResourceGroupName: '$(resourceGroup)' 
              backendAzureRmStorageAccountName: '$(storageAccount)'
              backendAzureRmStorageAccountSku: '$(storageAccountSku)'
              backendAzureRmContainerName: '$(container)'
              backendAzureRmKey: '$(tfstateFile)'
# Terraform Apply:-
          - task: TerraformCLI@0
            displayName: TERRAFORM APPLY 
            inputs:
              command: 'apply'
              backendType: 'azurerm'
              workingDirectory: '$(System.ArtifactsDirectory)/$(artifact)/AMTF'
              commandOptions: '--var-file=loga.tfvars'
              environmentServiceName: '$(ServiceConnection)'

DEPLOY STAGE PERFORMS BELOW:-

##	TASKS
1.	DEPLOY Stage will Execute only if the following conditions are met - 1) BUILD Stage gets completed successfully. 2) Option Deploy is selected from DevOps Runtime Parameters 3) Source Branch = Main. If not, DEPLOY Stage will get Skipped Automatically.
2.	DEPLOY Stage will Execute only after Approval. The Approval is integrated with Environment defined in the Pipeline Parameters Section and applied in Deploy Stage.
3.	Download the Published Artifacts.
4.	Terraform Installer installed in Azure DevOps Build Agent.
5.	Terraform Init
6.	Terraform Apply

DETAILS AND ALL TERRAFORM CODE SNIPPETS FOLLOWS BELOW:-

TERRAFORM (main.tf):-



terraform {
  required_version = ">= 1.2.0"

   backend "azurerm" {
    resource_group_name  = "tfpipeline-rg"
    storage_account_name = "tfpipelinesa"
    container_name       = "terraform"
    key                  = "PUBLISH-TF-PLAN/LogaPublishTFPlan.tfstate"
  }
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.2"
    }   
  }
}
provider "azurerm" {
  features {}
  skip_provider_registration = true
}

TERRAFORM (loga.tf):-



## Azure Resource Group:-
resource "azurerm_resource_group" "rg" {
  name     = var.rg-name
  location = var.rg-location
}

## Azure log Analytics Workspace:-

resource "azurerm_log_analytics_workspace" "loga" {
  name                = var.loga-name
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  sku                 = var.loga-sku 
  retention_in_days   = var.loga-retention

  depends_on          = [azurerm_resource_group.rg]
}

TERRAFORM (variables.tf):-



variable "rg-name" {
  type        = string
  description = "Name of the Resource Group"
}

variable "rg-location" {
  type        = string
  description = "Resource Group Location"
}

variable "loga-name" {
  type        = string
  description = "Name of the Log Analytics Workspace"
}

variable "loga-sku" {
  type        = string
  description = "SKU the Log Analytics Workspace"
}

variable "loga-retention" {
  type        = string
  description = "Retention Period of the Log Analytics Workspace"
}

TERRAFORM (loga.tfvars):-



rg-name         = "AMTESTRG100"
rg-location     = "West Europe"
loga-name       = "AMLOGA100"
loga-sku        = "PerGB2018"
loga-retention  = "30"

ITS TIME TO TEST:-
DESIRED RESULT: Stages - PUBLISH_PLAN, BUILD and DEPLOY should Complete Successfully. Terraform Plan Gets Published Successfully in Azure DevOps GUI. Resource Group and Log Analytics Workspace Resources gets deployed.
PIPELINE RUNTIME PARAMETERS WITH POPULATED VALUES:-

PIPELINE STAGE PUBLISH_PLAN EXECUTED SUCCESSFULLY:-

TERRAFORM PLAN GETS PUBLISHED IN AZURE DEVOPS GUI:-

PIPELINE STAGE BUILD EXECUTED SUCCESSFULLY:-

PIPELINE STAGE DEPLOY WAITING APPROVAL:-


PIPELINE STAGE DEPLOY EXECUTED SUCCESSFULLY:-

PIPELINE OVERALL EXECUTION STATUS:-


VALIDATE RESOURCES DEPLOYED IN PORTAL:-

IMPORTANT TO NOTE:-
Terraform Plan are NOT PUBLISHED in Azure DevOps GUI unless there is a change in Infrastructure - ADD, DESTROY or CHANGE
In order to demonstrate, the same pipeline was re-run.