Break Terraform state lease using Azure DevOps

#azure #cloud #devops #pipelines

Greetings to my fellow Technology Advocates and Specialists.

In this Session, I will demonstrate how to Break Terraform State Lease Using Azure DevOps.

USE CASE:-
In Order to Protect State File from Accidental Deletion or Tampering, Direct User Access to Terraform State File is Prohibited.
While Build IaC [Infrastructure-As-Code] Using Terraform, DevOps Engineer tend to Run the Code locally by manually executing Terraform Init, Plan and Apply Commands respectively.
During this whole Process, there might be Situation, where the Terraform State file is in Locked State and Unless the Lock is released, the code cannot be executed anymore (Manually or using Az DevOps Pipeline).
This is where, the below Az DevOps Pipeline helps.
The Az DevOps Pipeline runs in the Build Agent using Az DevOps Service Connection which is Az Service Principal Credentials behind the Scene with Appropriate RBAC [Role Based Access Control] on Subscription or Resource Group Level.

AUTOMATION OBJECTIVE:-
Validate If Resource Group Exists.
Validate If Storage Account Exists.
Validate If Storage Account Container Exists.
Validate If Terraform State File Exists in the Specified Storage Account Container.
If any One of the above validation DOES NOT PASS, Pipeline will Fail immediately.
If All of the above validation is SUCCESSFUL, Pipeline will then check the Terraform Blob State.
If Terraform Blob State is == LEASED, Pipeline will Break the Lease.
If Terraform Blob State is != LEASED, Pipeline Still executes Successfully without altering the present state.

IMPORTANT TO NOTE:-
There is No way to find the Blob Lease State before executing the az storage blob lease break command.
Refer the Link for more Information: az cli blob lease break

REQUIREMENTS:-

Azure Subscription.
Azure DevOps Organisation and Project.
Service Principal with Required RBAC ( Contributor) applied on Subscription or Resource Group(s).
Azure Resource Manager Service Connection in Azure DevOps.
Microsoft DevLabs Terraform Extension Installed in Azure DevOps and in Local System (VS Code Extension).

CODE REPOSITORY:-

arindam0310018 / 03-Aug-2022-DevOps__Break-Terraform-State-Lease

BREAK TERRAFORM STATE LEASE USING AZURE DEVOPS

Greetings to my fellow Technology Advocates and Specialists.

In this Session, I will demonstrate how to Break Terraform State Lease Using Azure DevOps.

USE CASE:-
In Order to Protect State File from Accidental Deletion or Tampering, Direct User Access to Terraform State File is Prohibited.
While Build IaC [Infrastructure-As-Code] Using Terraform, DevOps Engineer tend to Run the Code locally by manually executing Terraform Init, Plan and Apply Commands respectively.
During this whole Process, there might be Situation, where the Terraform State file is in Locked State and Unless the Lock is released, the code cannot be executed anymore (Manually or using Az DevOps Pipeline).
This is where, the below Az DevOps Pipeline helps.
The Az DevOps Pipeline runs in the Build Agent using Az DevOps Service Connection which is Az Service Principal Credentials behind the Scene with Appropriate RBAC [Role Based Access Control]

…

View on GitHub

HOW DOES MY CODE PLACEHOLDER LOOKS LIKE:-

PIPELINE CODE SNIPPET:-

AZURE DEVOPS YAML PIPELINE (azure-pipelines-tf-break-lease-v1.0.yml):-

trigger:
  none

######################
#DECLARE PARAMETERS:-
######################
parameters:
- name: SubscriptionID
  displayName: Subscription ID Details Follow Below:-
  type: string
  default: 210e66cb-55cf-424e-8daa-6cad804ab604
  values:
  - 210e66cb-55cf-424e-8daa-6cad804ab604

- name: RGNAME
  displayName: Please Provide the Resource Group Name:-
  type: object
  default: 

- name: STORAGEACCOUNTNAME
  displayName: Please Provide the Storage Account Name:-
  type: object
  default: 

- name: STORAGEACCOUNTCONTAINERNAME
  displayName: Please Provide the Storage Account Container Name:-
  type: object
  default:

- name: TFSTATEFILENAME
  displayName: Please Provide the TF State Name (For Example - "abc.tfstate" OR "Folder-A\abc.tfstate"):-
  type: object
  default: 

######################
#DECLARE VARIABLES:-
######################
variables:
  ServiceConnection: amcloud-cicd-service-connection
  BuildAgent: windows-latest

#########################
# Declare Build Agents:-
#########################
pool:
  vmImage: $(BuildAgent)

###################
# Declare Stages:-
###################

stages:

- stage: BREAK_TF_LEASE 
  jobs:
  - job: BREAK_TF_LEASE 
    displayName: BREAK TERRAFORM LEASE
    steps:
    - task: AzureCLI@2
      displayName: VALIDATE AND BREAK LEASE
      inputs:
        azureSubscription: $(ServiceConnection)
        scriptType: ps
        scriptLocation: inlineScript
        inlineScript: |
          az --version
          az account set --subscription ${{ parameters.SubscriptionID }}
          az account show  
          $i = az group exists -n ${{ parameters.RGNAME }}
            if ($i -eq "true") {
              echo "#####################################################"
              echo "Resource Group ${{ parameters.RGNAME }} exists!!!"
              echo "#####################################################"
              $j = az storage account check-name --name ${{ parameters.STORAGEACCOUNTNAME }} --query "reason" --out tsv
                if ($j -eq "AlreadyExists") {
                  echo "#####################################################"
                  echo "Storage Account ${{ parameters.STORAGEACCOUNTNAME }} exists!!!"
                  echo "#####################################################"  
                  $k = az storage container exists --account-name ${{ parameters.STORAGEACCOUNTNAME }} --account-key $(az storage account keys list -g ${{ parameters.RGNAME }} -n ${{ parameters.STORAGEACCOUNTNAME }} --query [0].value -o tsv) --name ${{ parameters.STORAGEACCOUNTCONTAINERNAME }} --query "exists" --out tsv 
                    if ($k -eq "true") {
                      echo "#####################################################"
                      echo "Storage Account Container ${{ parameters.STORAGEACCOUNTCONTAINERNAME }} exists!!!"
                      echo "#####################################################"  
                      $l = az storage blob exists --account-name ${{ parameters.STORAGEACCOUNTNAME }} --account-key $(az storage account keys list -g ${{ parameters.RGNAME }} -n ${{ parameters.STORAGEACCOUNTNAME }} --query [0].value -o tsv) --container-name ${{ parameters.STORAGEACCOUNTCONTAINERNAME }} --name ${{ parameters.TFSTATEFILENAME }} --query "exists" --out tsv  
                        if ($l -eq "true") {
                          echo "#####################################################"
                          echo "Terraform State Blob ${{ parameters.TFSTATEFILENAME }} exists!!!"
                          echo "#####################################################"
                          az storage blob lease break --account-name ${{ parameters.STORAGEACCOUNTNAME }} --account-key $(az storage account keys list -g ${{ parameters.RGNAME }} -n ${{ parameters.STORAGEACCOUNTNAME }} --query [0].value -o tsv) --container-name ${{ parameters.STORAGEACCOUNTCONTAINERNAME }} --blob-name ${{ parameters.TFSTATEFILENAME }}                           
                          }
                        else {
                          echo "#####################################################"
                          echo "Terraform State Blob ${{ parameters.TFSTATEFILENAME }} DOES NOT EXISTS in ${{ parameters.STORAGEACCOUNTCONTAINERNAME }}"
                          echo "#####################################################"
                          exit 1
                          }
                    }
                    else {
                      echo "#####################################################"
                      echo "Storage Account Container ${{ parameters.STORAGEACCOUNTCONTAINERNAME }} DOES NOT EXISTS in STORAGE ACCOUNT ${{ parameters.STORAGEACCOUNTNAME }}!!!"
                      echo "#####################################################"
                      exit 1
                    }
                }
                else {
                  echo "#####################################################"
                  echo "Storage Account ${{ parameters.STORAGEACCOUNTNAME }} DOES NOT EXISTS!!!"
                  echo "#####################################################"
                  exit 1
                }              
            }
            else {
              echo "#####################################################"
              echo "Resource Group ${{ parameters.RGNAME }} DOES NOT EXISTS!!!"
              echo "#####################################################"
              exit 1
            }

OBJECTIVE OF TERRAFORM CODE SNIPPET:-
Create a Resource Group and User Assigned System Managed Identity.
The Purpose of the Terraform Code Snippet is to Reproduce the Issue, by Locking Terraform State File .

TERRAFORM (main.tf):-

terraform {
  required_version = ">= 1.2.3"

   backend "azurerm" {
    resource_group_name  = "tfpipeline-rg"
    storage_account_name = "tfpipelinesa"
    container_name       = "terraform"
    key                  = "TF-LEASE/BreakLease.tfstate"
  }
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.2"
    }   
  }
}
provider "azurerm" {
  features {}
  skip_provider_registration = true
}

TERRAFORM (usrmid.tf):-

## Azure Resource Group:-
resource "azurerm_resource_group" "rg" {
 name     = var.rg-name
 location = var.rg-location
}

## Azure User Assigned Managed Identities:-
resource "azurerm_user_assigned_identity" "az-usr-mid" {

 name                = var.usr-mid-name
 resource_group_name = azurerm_resource_group.rg.name
 location            = azurerm_resource_group.rg.location

 depends_on          = [azurerm_resource_group.rg]
 }

TERRAFORM (variables.tf):-

variable "rg-name" {
  type        = string
  description = "Name of the Resource Group"
}

variable "rg-location" {
  type        = string
  description = "Resource Group Location"
}

variable "usr-mid-name" {
  type        = string
  description = "Name of the User Assigned Managed Identity"
}

TERRAFORM (usrmid.tfvars):-

rg-name         = "AMTest100"
rg-location     = "West Europe"
usr-mid-name    = "AMUSRMID100"

HOW TO LOCK TERRAFORM STATE FILE:-

Run the Terraform Apply Command manually in your local System as mentioned below:-

terraform apply --var-file="usrmid.tfvars"

When Prompted for "Yes", Press Control + C to terminate the Execution.

Next time, when you Re-Execute the Command, It will Inform the User that Terraform State File is in Locked State.

NOW ITS TIME TO TEST !!!...

TEST CASES:-

TEST CASE #1: WRONG OR NO RESOURCE GROUP NAME PROVIDED:-
DESIRED OUTPUT: PIPELINE WILL FAIL DISPLAYING CUSTOM MESSGAGE.
PIPELINE RUN:-

TEST CASE #2: WRONG OR NO STORAGE ACCOUNT NAME PROVIDED:-
DESIRED OUTPUT: PIPELINE WILL FAIL DISPLAYING CUSTOM MESSGAGE.
PIPELINE RUN:-

TEST CASE #3: WRONG OR NO STORAGE ACCOUNT CONTAINER NAME PROVIDED:-
DESIRED OUTPUT: PIPELINE WILL FAIL DISPLAYING CUSTOM MESSGAGE.
PIPELINE RUN:-

TEST CASE #4: WRONG OR NO TERRAFORM STATE FILE NAME PROVIDED:-
DESIRED OUTPUT: PIPELINE WILL FAIL DISPLAYING CUSTOM MESSGAGE.
PIPELINE RUN:-

TEST CASE #5: TERRAFORM STATE FILE FOUND == LEASED:-
DESIRED OUTPUT: PIPELINE WILL RUN SUCCESSFULLY BREAKING THE TERRAFORM BLOB LEASED STATE.
PIPELINE RUN:-




TEST CASE #6: TERRAFORM STATE FILE FOUND != LEASED:-
DESIRED OUTPUT: PIPELINE STILL EXECUTES SUCCESSFULLY WITHOUT ALTERING THE PRESENT STATE.
PIPELINE RUN:-