Greetings my fellow Technology Advocates and Specialists.
In this Session, I will demonstrate -
- How to Map Azure DevOps Runtime Variables to Terraform Input Variables.
- If at all we need to put the values in variables.tf or in tfvars.
I had the Privilege to talk on this topic in ONE Azure Communities:-
NAME OF THE AZURE COMMUNITY | TYPE OF SPEAKER SESSION |
---|---|
Boston Azure User Group | Virtual |
EVENT ANNOUNCEMENT:- |
---|
LIVE RECORDED SESSIONS:- |
---|
LIVE DEMO was Recorded as part of my Presentation in BOSTON AZURE USER GROUP Forum/Platform |
Duration of My Demo = 41 Mins 07 Secs |
REQUIREMENTS:- |
---|
- Azure Subscription.
- Azure DevOps Organisation and Project.
- Service Principal with Delegated Graph API Rights and Required RBAC (Typically Contributor on Subscription or Resource Group)
- Azure Resource Manager Service Connection in Azure DevOps.
- Microsoft DevLabs Terraform Extension Installed in Azure DevOps.
CODE REPOSITORY:- | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
arindam0310018 / 23-May-2022-DevOps__Runtime-Variables-To-Terraform-Input-VariablesMAP AZURE DEVOPS RUNTIME VARIABLES TO TERRAFORM INPUT VARIABLESMAP AZURE DEVOPS RUNTIME VARIABLES TO TERRAFORM INPUT VARIABLESGreetings my fellow Technology Advocates and Specialists. In this Session, I will demonstrate -
I had the Privilege to talk on this topic in ONE Azure Communities:-
|
HOW DOES MY CODE PLACEHOLDER LOOKS LIKE:- |
---|
OBJECTIVE:- |
---|
Deploy a Resource Group and User Assigned Managed Identity from the values provided by user in the DevOps Runtime Variables Parameters and not providing it again in Terraform variables.tf or tfvars |
PIPELINE CODE SNIPPET:- |
---|
AZURE DEVOPS YAML PIPELINE (azure-pipelines-usr-mid-v1.0.yml):- |
---|
trigger:
none
######################
#DECLARE PARAMETERS:-
######################
parameters:
- name: SubscriptionID
displayName: Subscription ID Details Follow Below:-
default: 210e66cb-55cf-424e-8daa-6cad804ab604
values:
- 210e66cb-55cf-424e-8daa-6cad804ab604
- name: ServiceConnection
displayName: Service Connection Name Follows Below:-
default: amcloud-cicd-service-connection
values:
- amcloud-cicd-service-connection
- name: RGNAME
displayName: Please Provide the Resource Group Name:-
type: object
default: <Please provide the required Name>
- name: USRMIDNAME
displayName: Please Provide the User Assigned Managed Identity Name:-
type: object
default: <Please provide the required Name>
######################
#DECLARE VARIABLES:-
######################
variables:
TF_VAR_RG_NAME: ${{ parameters.RGNAME }}
TF_VAR_USR_MID_NAME: ${{ parameters.USRMIDNAME }}
ResourceGroup: tfpipeline-rg
StorageAccount: tfpipelinesa
Container: terraform
TfstateFile: UMID/usrmid.tfstate
BuildAgent: windows-latest
WorkingDir: $(System.DefaultWorkingDirectory)/Usr-MID
Target: $(build.artifactstagingdirectory)/AMTF
Environment: NonProd
Artifact: AM
#########################
# Declare Build Agents:-
#########################
pool:
vmImage: $(BuildAgent)
###################
# Declare Stages:-
###################
stages:
- stage: PLAN
jobs:
- job: PLAN
displayName: PLAN
steps:
# Install Terraform Installer in the Build Agent:-
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
displayName: INSTALL TERRAFORM VERSION - LATEST
inputs:
terraformVersion: 'latest'
# Terraform Init:-
- task: TerraformTaskV2@2
displayName: TERRAFORM INIT
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDir)' # Az DevOps can find the required Terraform code
backendServiceArm: '${{ parameters.ServiceConnection }}'
backendAzureRmResourceGroupName: '$(ResourceGroup)'
backendAzureRmStorageAccountName: '$(StorageAccount)'
backendAzureRmContainerName: '$(Container)'
backendAzureRmKey: '$(TfstateFile)'
# Terraform Validate:-
- task: TerraformTaskV2@2
displayName: TERRAFORM VALIDATE
inputs:
provider: 'azurerm'
command: 'validate'
workingDirectory: '$(workingDir)'
environmentServiceNameAzureRM: '${{ parameters.ServiceConnection }}'
# Terraform Plan:-
- task: TerraformTaskV2@2
displayName: TERRAFORM PLAN
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(workingDir)'
commandOptions: "--var-file=usrmid.tfvars --out=tfplan"
environmentServiceNameAzureRM: '${{ parameters.ServiceConnection }}'
# Copy Files to Artifacts Staging Directory:-
- task: CopyFiles@2
displayName: COPY FILES ARTIFACTS STAGING DIRECTORY
inputs:
SourceFolder: '$(workingDir)'
Contents: |
**/*.tf
**/*.tfvars
**/*tfplan*
TargetFolder: '$(Target)'
# Publish Artifacts:-
- task: PublishBuildArtifacts@1
displayName: PUBLISH ARTIFACTS
inputs:
targetPath: '$(Target)'
artifactName: '$(Artifact)'
- stage: DEPLOY
condition: succeeded()
dependsOn: PLAN
jobs:
- deployment:
displayName: Deploy
environment: $(Environment)
pool:
vmImage: '$(BuildAgent)'
strategy:
runOnce:
deploy:
steps:
# Download Artifacts:-
- task: DownloadBuildArtifacts@0
displayName: DOWNLOAD ARTIFACTS
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: '$(Artifact)'
downloadPath: '$(System.ArtifactsDirectory)'
# Install Terraform Installer in the Build Agent:-
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
displayName: INSTALL TERRAFORM VERSION - LATEST
inputs:
terraformVersion: 'latest'
# Terraform Init:-
- task: TerraformTaskV2@2
displayName: TERRAFORM INIT
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(System.ArtifactsDirectory)/$(Artifact)/AMTF/' # Az DevOps can find the required Terraform code
backendServiceArm: '${{ parameters.ServiceConnection }}'
backendAzureRmResourceGroupName: '$(ResourceGroup)'
backendAzureRmStorageAccountName: '$(StorageAccount)'
backendAzureRmContainerName: '$(Container)'
backendAzureRmKey: '$(TfstateFile)'
# Terraform Apply:-
- task: TerraformTaskV2@2
displayName: TERRAFORM APPLY # The terraform Plan stored earlier is used here to apply only the changes.
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(System.ArtifactsDirectory)/$(Artifact)/AMTF'
commandOptions: '--var-file=usrmid.tfvars'
environmentServiceNameAzureRM: '${{ parameters.ServiceConnection }}'
Now, let me explain each part of YAML Pipeline for better understanding.
PART #1:- |
---|
BELOW FOLLOWS PIPELINE RUNTIME VARIABLES CODE SNIPPET:- |
---|
######################
#DECLARE PARAMETERS:-
######################
parameters:
- name: SubscriptionID
displayName: Subscription ID Details Follow Below:-
default: 210e66cb-55cf-424e-8daa-6cad804ab604
values:
- 210e66cb-55cf-424e-8daa-6cad804ab604
- name: ServiceConnection
displayName: Service Connection Name Follows Below:-
default: amcloud-cicd-service-connection
values:
- amcloud-cicd-service-connection
- name: RGNAME
displayName: Please Provide the Resource Group Name:-
type: object
default: <Please provide the required Name>
- name: USRMIDNAME
displayName: Please Provide the User Assigned Managed Identity Name:-
type: object
default: <Please provide the required Name>
THIS IS HOW IT LOOKS WHEN YOU EXECUTE THE PIPELINE FROM AZURE DEVOPS:- |
---|
NOTE:- |
---|
Please Provide the Name of the Resource Group |
For Example: AMTESTMIDRG |
Please Provide the Name of the User Assigned Managed Identity |
For Example: AMMID100 |
PART #2:- |
---|
BELOW FOLLOWS PIPELINE VARIABLES CODE SNIPPET:- |
---|
######################
#DECLARE VARIABLES:-
######################
variables:
TF_VAR_RG_NAME: ${{ parameters.RGNAME }}
TF_VAR_USR_MID_NAME: ${{ parameters.USRMIDNAME }}
ResourceGroup: tfpipeline-rg
StorageAccount: tfpipelinesa
Container: terraform
TfstateFile: UMID/usrmid.tfstate
BuildAgent: windows-latest
WorkingDir: $(System.DefaultWorkingDirectory)/Usr-MID
Target: $(build.artifactstagingdirectory)/AMTF
Environment: NonProd
Artifact: AM
IMPORTANT TO NOTE:- |
---|
User Input Values from DevOps Runtime Parameters are referenced to DevOps Variables. |
Notice the the variables TF_VAR_RG_NAME and TF_VAR_USR_MID_NAME. |
Azure DevOps Variables gets automatically mapped to Environment Variables in Azure DevOps Build Agent. |
Environment Variables which Starts with TF_VAR_ gets automatically mapped to Terraform Input Variables |
Refer the link to find more: https://www.terraform.io/cli/config/environment-variables |
GENERAL INFORMATION:- |
---|
Please feel free to change the values of the variables. |
The entire YAML pipeline is build using Parameters and variables. No Values are Hardcoded. |
PART #3:- |
---|
PIPELINE STAGE DETAILS FOLLOW BELOW:- |
---|
- This is a Two Stage Pipeline with 4 Runtime Variables - 1) Subscription ID 2) Service Connection Name 3) Resource Group Name and 4) User Assigned Managed Identity Name
- The Names of the Stages are - 1) PLAN and 2) DEPLOY
PIPELINE STAGE - PLAN:- |
---|
- stage: PLAN
jobs:
- job: PLAN
displayName: PLAN
steps:
# Install Terraform Installer in the Build Agent:-
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
displayName: INSTALL TERRAFORM VERSION - LATEST
inputs:
terraformVersion: 'latest'
# Terraform Init:-
- task: TerraformTaskV2@2
displayName: TERRAFORM INIT
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDir)' # Az DevOps can find the required Terraform code
backendServiceArm: '${{ parameters.ServiceConnection }}'
backendAzureRmResourceGroupName: '$(ResourceGroup)'
backendAzureRmStorageAccountName: '$(StorageAccount)'
backendAzureRmContainerName: '$(Container)'
backendAzureRmKey: '$(TfstateFile)'
# Terraform Validate:-
- task: TerraformTaskV2@2
displayName: TERRAFORM VALIDATE
inputs:
provider: 'azurerm'
command: 'validate'
workingDirectory: '$(workingDir)'
environmentServiceNameAzureRM: '${{ parameters.ServiceConnection }}'
# Terraform Plan:-
- task: TerraformTaskV2@2
displayName: TERRAFORM PLAN
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(workingDir)'
commandOptions: "--var-file=usrmid.tfvars --out=tfplan"
environmentServiceNameAzureRM: '${{ parameters.ServiceConnection }}'
# Copy Files to Artifacts Staging Directory:-
- task: CopyFiles@2
displayName: COPY FILES ARTIFACTS STAGING DIRECTORY
inputs:
SourceFolder: '$(workingDir)'
Contents: |
**/*.tf
**/*.tfvars
**/*tfplan*
TargetFolder: '$(Target)'
# Publish Artifacts:-
- task: PublishBuildArtifacts@1
displayName: PUBLISH ARTIFACTS
inputs:
targetPath: '$(Target)'
artifactName: '$(Artifact)'
PLAN STAGE PERFORMS BELOW:- |
---|
## | TASKS |
---|---|
1. | Terraform Installer installed in Azure DevOps Build Agent. |
2. | Terraform Init |
3. | Terraform Validate |
4. | Terraform Plan |
5. | Copy the Terraform files (Most Importantly Terraform Plan Output) to Artifacts Staging Directory. |
6. | Publish Artifacts |
PIPELINE STAGE - DEPLOY:- |
---|
- stage: DEPLOY
condition: succeeded()
dependsOn: PLAN
jobs:
- deployment:
displayName: Deploy
environment: $(Environment)
pool:
vmImage: '$(BuildAgent)'
strategy:
runOnce:
deploy:
steps:
# Download Artifacts:-
- task: DownloadBuildArtifacts@0
displayName: DOWNLOAD ARTIFACTS
inputs:
buildType: 'current'
downloadType: 'single'
artifactName: '$(Artifact)'
downloadPath: '$(System.ArtifactsDirectory)'
# Install Terraform Installer in the Build Agent:-
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
displayName: INSTALL TERRAFORM VERSION - LATEST
inputs:
terraformVersion: 'latest'
# Terraform Init:-
- task: TerraformTaskV2@2
displayName: TERRAFORM INIT
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(System.ArtifactsDirectory)/$(Artifact)/AMTF/' # Az DevOps can find the required Terraform code
backendServiceArm: '${{ parameters.ServiceConnection }}'
backendAzureRmResourceGroupName: '$(ResourceGroup)'
backendAzureRmStorageAccountName: '$(StorageAccount)'
backendAzureRmContainerName: '$(Container)'
backendAzureRmKey: '$(TfstateFile)'
# Terraform Apply:-
- task: TerraformTaskV2@2
displayName: TERRAFORM APPLY # The terraform Plan stored earlier is used here to apply only the changes.
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(System.ArtifactsDirectory)/$(Artifact)/AMTF'
commandOptions: '--var-file=usrmid.tfvars'
environmentServiceNameAzureRM: '${{ parameters.ServiceConnection }}'
DEPLOY STAGE PERFORMS BELOW:- |
---|
## | TASKS |
---|---|
1. | DEPLOY Stage will Execute only if PLAN Stage completed successfully. If not, DEPLOY Stage will get Skipped Automatically. |
2. | DEPLOY Stage will Execute only after Approval. The Approval is integrated with Environment defined in the Pipeline Variable Section (Environment: NonProd) and applied in DEPLOY Stage Jobs (environment: $(Environment)). |
3. | Download the Published Artifacts. |
4. | Terraform Installer installed in Azure DevOps Build Agent. |
5. | Terraform Init |
6. | Terraform Apply |
DETAILS AND ALL TERRAFORM CODE SNIPPETS FOLLOWS BELOW:- |
---|
TERRAFORM (main.tf):- |
---|
terraform {
required_version = ">= 1.2.0"
backend "azurerm" {
resource_group_name = "tfpipeline-rg"
storage_account_name = "tfpipelinesa"
container_name = "terraform"
key = "UMID/usrmid.tfstate"
}
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.2"
}
}
}
provider "azurerm" {
features {}
skip_provider_registration = true
}
TERRAFORM (usrmid.tf):- |
---|
## Azure Resource Group:-
resource "azurerm_resource_group" "rg" {
name = var.RG_NAME
location = var.rg-location
}
## Azure User Assigned Managed Identities:-
resource "azurerm_user_assigned_identity" "az-usr-mid" {
name = var.USR_MID_NAME
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
depends_on = [azurerm_resource_group.rg]
}
TERRAFORM (variables.tf):- |
---|
variable "RG_NAME" {
type = string
description = "Name of the Resource Group"
}
variable "rg-location" {
type = string
description = "Resource Group Location"
}
variable "USR_MID_NAME" {
type = string
description = "Name of the User Assigned Managed Identity"
}
IMPORTANT TO NOTE:- |
---|
The Variable name of the Resource Group and User Assigned Managed Identities in usrmid.tf and variables.tf are in upper case. |
This is because Azure DevOps Pipeline variables which automatically references to Build Agent Environment Variables gets converted to uppercase |
If the variables are not defined as above, the Pipeline waits for Resource Group and User Assigned Managed Identity Name as Input. |
The Pipeline is then cancelled manually |
TERRAFORM (usrmid.tfvars):- |
---|
rg-location = "West Europe"
IMPORTANT TO NOTE:- |
---|
There is No Resource Group and User Assigned Managed Identity Name Value provided in tfvars or in variables.tf |
ITS TIME TO TEST:- |
---|
DESIRED RESULT: Stages - PLAN and DEPLOY should Complete Successfully. Resource Group and User Assigned Managed Identity Resources should get deployed. Remote State file gets created. |
PIPELINE RUNTIME PARAMETERS WITH POPULATED VALUES:- |
PIPELINE STAGE PLAN EXECUTED SUCCESSFULLY:- |
PIPELINE STAGE DEPLOY WAITING APPROVAL:- |
PIPELINE STAGE DEPLOY EXECUTED SUCCESSFULLY:- |
PIPELINE OVERALL EXECUTION STATUS:- |
VALIDATE RESOURCES DEPLOYED IN PORTAL:- |
VALIDATE REMOTE TERRAFORM STATE FILE:- |
Top comments (0)