Infra as Code in Azure, what's new for 2020 in ARM Templates deployment

During the MS Ignite 2019 in November, there were some big announcements about new features in ARM Templates deployments. As I mostly work with ARM Template when doing infrastructure as Code, I choose to talks about a few of them.

Comments

For log, I thought the only way to comment on something was to use the COMMENTS properties in a resource.

"comments": "This storage account used by the function App",

But in fact, I was wrong. ARM Templates support /* */ and // comments for long.

  "variables": {
   // "storageAccount": "[concat('st', uniquestring(resourceGroup().id))]"
  },
  "resources": [
    /*
    {
      "type": "Microsoft.Storage/storageAccounts",
      "name": "[variables('storageAccount')]",
      "location": "[parameters('location')]",
      "comments": "",
      "apiVersion": "2019-04-01",
      "sku": {
        "name": "[parameters('storageAccountType')]"
      },
      "kind": "StorageV2",
      "properties": {}
    }
    */  
  ],

You may have some errors in VSCode or other tools before. But If you use VSCode with the latest version of Microsoft Azure ARM Tools it should not be a problem now and you can start commenting one line or an entire resource (just take care about comma)

There are 3 valid schema for ARM deployments.

• 2014-04-01-preview
• 2015-05-01
• 2019-04-01

Only the latest, 2019-04-01 will receive schema update for new Azure resources and API version.

With the updated version of ARM Tools for VsCode, There were 3 other major announcements at the MS Ignite 2019. What-IF, The DeployementScripts resource, and the ARM Toolkit. These news features are still in preview and you will have to request access via this link.

What IF

When I think about what-if, I think about the what-if switch you can use with a PowerShell Cmdlet. The option gives you the choice to run a command and to see the action the command will perform if you run it.
The purpose of the what-if feature in Azure is to tell what will be deployed, updated or removed by a template file. This feature is similar to what you get when you use the Terraform Plan command. It gives you hints about what the template will to your environment in Azure.

The tool is based on a PowerShell cmdlet, new-AzDeploymentWhatif. The cmdlet needs to know the scope, either a resource group or a subscription, the resource group name (or the subscription name) and the path to the template file you need to test.
It will list all the resources in the template, compare the result list with the current state of resources within the scope, either the subscriptions or the resource group.

New-AzDeploymentWhatIf -ScopeType ResourceGroup -ResourceGroupName MyRg -TemplateFile .\mytemplate.json

If there is nothing in the current resource group, the desired state will be a simple resource creation and the cmdlet will color resource to be created in green with the properties resolved. But if a resource must be deleted it will show up in red and if a property in a resource has to be modified it will show up in purple.

This feature is still in private preview and you need to request access via this link and you may encounter some false positives. you can report them on GitHub it will help to reduce the noise for the final product.

DeployementsScripts Resource Provider

This is a new resource that will help you to go beyond ARM deployment, Microsoft.Resources/deploymentsScripts. It’s in preview, the API is 2019-10-01-preview.

The Deployment script resource provider will take a script and parameters and will build a container (with PowerShell Core and Azure PowerShell) in the current subscription. A managed identity can be associated with the container to perform actions in the subscription.
this resource can be seen as the script extension for a VM at the subscription level. The script VM extension lets you configure something in the VM after the VM deployment.

DeploymentsScripts must be used in this way. Extending your ARM deployments with actions you cannot do with ARM deployments alone. If a template pop up a storage account, you may want to download files inside, you may need to do an App registration, an Azure Ad setup or simply perform some tests.

But, deployments Scripts resource should not be used to deploy other Azure resources. It will be weird to use it to deploy a VM. Doing this will make your deployment imperative (versus declarative) and non-idempotent. It will break the rules of Infrastructure as Code, being able to deploy resources many times the same template and only modify, create or delete when there is a drift between the JSON document and deployed resources.

DeploymentsScripts resource is a run and forgets process. It will be your responsibility to make the script idempotent (test the desired state before making any modification) and testable.

How does it work? The resource will pop a container in Azure using Azure Container Instance and a Storage Account with the AzureDeploymentscript-PowerShell docker image.
The resource takes several parameters. Among them, you need to provide an identity with

"identity": {
                "type": "UserAssigned",
                "userAssignedIdentities": {
                    "[variables('Identity')]": {
                    }
                }
            }

This identity needs to be configured in Azure to guaranty access to the components you need.
One of the most important properties is forceUpdateTag. When you deploy a resource with ARM, the process looks at the current state, but the DeploymentsScripts resource will always look the same (same scripts, same parameters...) so it will not be re-deployed. To avoid this situation we need to change the resource tag for every deployment.

"forceUpdateTag": "[newGuid()]", 

You can pass arguments to the script in the container by using the arguments property.

"arguments": "[format(' -testname {0} ', variables('TestName'))]", 

You may also need to create environment variables in the container to store sensitive information.

"environmentVariables": [
                    {
                        "name": "myenv",
                        "secureValue": "My Environment variable"
                    }
                ]

You can control the timeout of the script with the Timeout property, so the container will shut down regardless of the outcome of the script after an amount of time. Here a ten minutes timeout.

"timeout": "PT10M",

For the PowerShell script it can be inline with the template file.

 "scriptContent": " write-host $testname
                $DeploymentScriptOutputs = $testname
                "

Or the script can be on a public url.

"PrimaryScriptUri": "http://myhost.com/myscript.ps1"

In this case, the URL must be public, or you need to provide the access credential or the token to Azure Resource Manager.
To obtain the output after the deployment
To return a value from the script to the template you need to use the $DeploymentScriptOutputs variable.

You can retrieve the output from the script resource in the Output section by referencing the deploymentsScrpts resource with outputs. You can also get the status of the execution with the status.

"outputs": {
        "testname": {
        "type": "string",
        "value": "[reference(variables('script').outputs)]"
        },
        "scriptOutput": {
            "type":"object",
            "value":"[reference(variables('script')).status]"
        }
    }

ARM Template Toolkit

Testing in infrastructure as code, as for any code, is essential. It helps to detect and correct errors before the deployment phase. There are some opensource initiatives in GitHub (From Chris Gardner or Barbara 4Bes) to perform tests or analyses on the code.

Microsoft launched a module to handle ARM templates tests, ARM-TTK or Azure Resource Manager Template Toolkit. The module is still in preview and you wouldn't be able to install it using install-module. You need to do that manually via GitHub.

It’s a static code analyzer and it doesn't rely on Azure API. The module is based on test cases, and it compares a template or a group of templates to theses test cases to enforce best practices and correct errors.
There are several test cases, like Location Should not be hardcoded, VM size should be in parameters or Secure string parameters should not have a default value.
As the module is not yet in the PowerShell Gallery, you need to download it from the Github Repository and import the module.

Import-module c:\xxx\azure-quickstart-templates\test\arm-ttk\arm-ttk.psd1

To test a template

Test-AzTemplate -TemplatePath x\template.json

To test an entire folder

Test-AzTemplate -TemplatePath x\arm

The module use Pester to perform tests. The output will be familiar if you use it too. But if you want a real pester result you should use the -Pester switch

Test-AzTemplate -TemplatePath x\template.json -Pester

This last option is better for pipeline scenarios.

Several other improvements will come in the next weeks. Like the new Environment() function ([environment().name]) to see if you are in AzureStack, Azure or Azure Government.

Thank Satya Vel for encouraged me to write this post.