DEV Community

Cover image for Building a CI/CD pipeline for your Azure project with GitHub Actions from scratch
Manuel Sidler
Manuel Sidler

Posted on • Edited on

Building a CI/CD pipeline for your Azure project with GitHub Actions from scratch

#github #azure

Cover photo by Quinten de Graaf on Unsplash

So you have decided to host your web application on Azure. Now it's time to take care of the infrastructure. Let's open the Azure portal, create all the resources and publish your application with Visual Studio.

But wait! You're going to add some resources and re-create the same infrastructure for different environments. Manual work all over, time-consuming, and prone to error. Wouldn't it be great to have infrastructure as code and let the CI/CD system continuously deploy it together with your web application?

In this blog post, we will use Azure Resource Manager templates and GitHub actions to achieve all of that.

Before we start, please be aware that this blog post isn't a theoretical introduction to ARM templates nor GitHub actions. Our goal is to set up a CI/CD pipeline from scratch to avoid right-click publishing from Visual Studio and manually create Azure resources. We're good people, and good people don't let friends right-click publishing their projects ;-)

If you're interested in Azure infrastructure as code in general, definitely check out other solutions as well like Terraform, Pulumi, Farmer and Bicep.

Tools

The following tools will help us to set up our pipeline:

Prerequisites

To set up a CI/CD pipeline for our Azure project, we need a GitHub repository, an active Azure subscription and a web application.

In the following example, we use a plain web application created by the dotnet CLI:

dotnet new webapp -o HelloWorld
Enter fullscreen mode Exit fullscreen mode

With that, our basic folder structure in our GitHub repository looks like the following:

│   .gitignore
│
└───src
    │
    └───web
        │   HelloWorld.csproj
        │   ...
Enter fullscreen mode Exit fullscreen mode

The last prerequisite is an active Azure subscription. We can use the Azure CLI to log in to our account:

az login
Enter fullscreen mode Exit fullscreen mode

After a successful login we can see our subscription details in the command prompt:

{
  "environmentName": "AzureCloud",
  "homeTenantId": "xxx-xxx-xxx-xxx-xxx",
  "id": "xxx-xxx-xxx-xxx-xxx",
  "isDefault": true,
  "managedByTenants": [],
  "name": "Azure subscription 1",
  "state": "Enabled",
  "tenantId": "xxx-xxx-xxx-xxx-xxx",
  "user": {
    "name": "manuel.sidler@xxx.xxx",
    "type": "user"
  }
}
Enter fullscreen mode Exit fullscreen mode

Now we're ready to go!

Azure infrastructure

First, we need to create an Azure resource group. It's a container to group different Azure resources together. We can use the Azure CLI to make the group:

az group create --location "West Europe" --name rg-dev-blog
Enter fullscreen mode Exit fullscreen mode

Please notice the abbreviation rg- in the name parameter. It's important to use some conventions when we name Azure resources. In case we don't have any specific ones defined in our company, we should stick to the abbreviations documented by Microsoft. If the command ran successful, we get a response with some details about the created resource group:

{
  "id": "/subscriptions/xxx-xxx-xxx-xxx-xxx/resourceGroups/rg-dev-blog",
  "location": "westeurope",
  "managedBy": null,
  "name": "rg-dev-blog",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}
Enter fullscreen mode Exit fullscreen mode

For our web application, we're going to set up three Azure resources:

  • Application Insights
  • App Service Plan
  • App Service

Application Insights is optional, but it's always a good idea to have it for logging, events, and other stuff. For more information, visit the official documentation from Microsoft.

The App Service Plan is kind of a server farm, which will host our App Service (the dotnet web application in this case). An App Service has to be assigned to an App Service Plan. If you're coming from a Windows Server world, you can think of an App Service Plan as an IIS and an app service as a site.

ARM Template

Instead of creating these three resources in the Azure portal, we define it in an ARM template JSON file. The question now is where to store it. As we use infrastructure as code here obviously, we should save it in our source directory:

│   ...
│
└───src
    │
    └───azure
    |   |   azuredeploy.json
    |
    └───web
        │   ...
Enter fullscreen mode Exit fullscreen mode

Thanks to the Azure Resource Manager (ARM) Tools extension for VS Code, we now can type arm! in the empty file. This snippet will create a skeleton of the ARM template:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {},
    "functions": [],
    "variables": {},
    "resources": [],
    "outputs": {}
}
Enter fullscreen mode Exit fullscreen mode

Now let's start with the Application Insights resource:

{
    ...
    "variables": {
        "appInsightsName": "appi-dev-blog"
    },
    "resources": [
        {
            "name": "[variables('appInsightsName')]",
            "type": "Microsoft.Insights/components",
            "apiVersion": "2015-05-01",
            "location": "[resourceGroup().location]",
            "kind": "web",
            "properties": {
                "application_Type": "web"
            }
        }
    ],
    ...
}
Enter fullscreen mode Exit fullscreen mode

There are two things to notice here:

  • Use variables for resource names, because we usually reference them in several places inside an ARM template
  • To avoid additional traffic cost and performance issues, host your Azure resources in the same location. We can use the resourceGroup().location function to refer to the resource group location

Next, we take care of the App Service Plan:

{
    ...
    "parameters": {
        "appServicePlanSku": {
            "type": "string"
        }
    },
    ...
    "variables": {
        ...
        "appServicePlanName": "plan-dev-blog"
    },
    "resources": [
        ...
        {
            "name": "[variables('appServicePlanName')]",
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "2018-02-01",
            "location": "[resourceGroup().location]",
            "sku": {
                "name": "[parameters('appServicePlanSku')]"
            },
            "properties": {

            }
        }
    ],
    ...
}
Enter fullscreen mode Exit fullscreen mode

Please notice the sku option, which basically defines the CPU and memory of our machine. This option will probably vary in different environments. We introduce a new parameter to pass it from outside into the template. We'll take care of that in a later step.

Last but not least, here's the template definition for our Azure App Service:

{
    ...
    "variables": {
        ...
        "appServiceName": "app-dev-blog"
    },
    "resources": [
        ...
        {
            "name": "[variables('appServiceName')]",
            "type": "Microsoft.Web/sites",
            "apiVersion": "2018-11-01",
            "location": "[resourceGroup().location]",
            "dependsOn": [
                "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
                "[resourceId('Microsoft.Insights/components', variables('appInsightsName'))]"
            ],
            "kind": "app",
            "properties": {
                "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
                "siteConfig": {
                    "appSettings": [
                        {
                            "name": "APPLICATIONINSIGHTS_CONNECTION_STRING",
                            "value": "[concat('InstrumentationKey=',reference(resourceId('Microsoft.Insights/components', variables('appInsightsName')), '2015-05-01').InstrumentationKey)]"
                        },
                        {
                            "name": "ApplicationInsightsAgent_EXTENSION_VERSION",
                            "value": "~2"
                        },
                        {
                            "name": "APPINSIGHTS_PROFILERFEATURE_VERSION",
                            "value": "1.0.0"
                        },
                        {
                            "name": "APPINSIGHTS_SNAPSHOTFEATURE_VERSION",
                            "value": "1.0.0"
                        },
                        {
                            "name": "DiagnosticServices_EXTENSION_VERSION",
                            "value": "~3"
                        },
                        {
                            "name": "InstrumentationEngine_EXTENSION_VERSION",
                            "value": "~1"
                        },
                        {
                            "name": "SnapshotDebugger_EXTENSION_VERSION",
                            "value": "~1"
                        },
                        {
                            "name": "XDT_MicrosoftApplicationInsights_BaseExtensions",
                            "value": "~1"
                        },
                        {
                            "name": "XDT_MicrosoftApplicationInsights_Mode",
                            "value": "recommended"
                        },
                        {
                            "name": "XDT_MicrosoftApplicationInsights_PreemptSdk",
                            "value": "disabled"
                        }
                    ]
                }
            }
        }
    ],
    ...
}
Enter fullscreen mode Exit fullscreen mode

A couple of things to mention here:

  • With dependsOn we're able to define dependencies between resources in an ARM template. When deploying the template, the Azure Resource Manager will take care of the dependency graph and create the Azure resources in the correct order
  • While dependsOn defines the relationship inside the template, serverFarmId sets the actual reference to the App Service Plan instance
  • We add several app settings entries to enable and configure the Application Insights options

ARM Template Parameters

Our ARM template is now complete. Or is it? Remember the sku parameter for the App Service Plan? Right! We have to create an additional parameters file:

│   ...
│
└───src
    │
    └───azure
    |   |   azuredeploy.json
    |   |   azuredeploy.parameters.json
    |
    └───web
        │   ...
Enter fullscreen mode Exit fullscreen mode

In a real project, we would create a parameters file per environment. But in our example here one is enough. The content of the file is actually quite simple. We can again use a snippet (armp!) to create the skeleton:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
    }
}
Enter fullscreen mode Exit fullscreen mode

Now we just have to define our sku parameter (visit the official documentation for all available plans):

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "appServicePlanSku": {
            "value": "F1"
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Azure Active Directory Service Principal

There's one thing left to do before we take care of the actual CI/CD pipeline. We need an Azure Active Directory service principal which we have to use for deploying the ARM template. Let's go back to the Azure CLI and do that:

az ad sp create-for-rbac --name sp-dev-blog --role contributor --scopes /subscriptions/xxx-xxx-xxx-xxx-xxx/resourceGroups/rg-dev-blog --sdk-auth
Enter fullscreen mode Exit fullscreen mode

When working in Azure, we should always stick with the principle of least privilege. That's why we assign the contributor role just to the scope of our resource group.

By setting the --sdk-auth parameter, we get back a JSON response which we have to save as a GitHub repository secret:

{
  "clientId": "xxx-xxx-xxx-xxx-xxx",
  "clientSecret": "xxx",
  "subscriptionId": "xxx-xxx-xxx-xxx-xxx",
  "tenantId": "xxx-xxx-xxx-xxx-xxx",
  "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
  "resourceManagerEndpointUrl": "https://management.azure.com/",
  "activeDirectoryGraphResourceId": "https://graph.windows.net/",
  "sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
  "galleryEndpointUrl": "https://gallery.azure.com/",
  "managementEndpointUrl": "https://management.core.windows.net/"
}
Enter fullscreen mode Exit fullscreen mode

To do that, we open our GitHub repository, switch to Settings, click Secrets and add the secret there:
GitHub secrets

We're done now with defining our Azure infrastructure as code. Perfect!

GitHub actions for Azure

GitHub Action workflows are defined in .yml files and stored under .github/workflows:

│   ...
│
│───.github
│   │
│   └───workflows
│       |   azure.yml
│       |   web.yml
|
└───src
    │
    └───azure
    |   |   ...
    |
    └───web
        │   ...
Enter fullscreen mode Exit fullscreen mode

So let's create those folders and the .yml files and start with azure.yml. First, we have to give the workflow a name and define a trigger:

name: Azure

on:
  push:
    paths:
      - 'src/azure/**'
      - '.github/workflows/azure.yml'
    branches:
      - '**'
Enter fullscreen mode Exit fullscreen mode

Since we have our web application and Azure infrastructure as code in the same repository, we don't want to trigger this workflow if something inside the web application changes. That's why we configure a path filter here.

Next, we define some environment variables which we later can reuse in our jobs:

...

env:
  AZURE_RESOURCE_GROUP: 'rg-dev-blog'
  TEMPLATE_FILE: 'src/azure/azuredeploy.json'
  PARAMETERS_FILE: 'src/azure/azuredeploy.parameters.json'
Enter fullscreen mode Exit fullscreen mode

Now we have to define the continuous integration and deployment jobs. For continuous integration, we'd like to execute the following steps:

  1. Checkout source code
  2. Log in to Azure CLI
  3. Validate the ARM template
  4. Execute a what-if command to see, what actually would happen in case of a deployment

And here's the result in our azure.yml file:

...

jobs:
  CI:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: login to Azure
        uses: Azure/login@v1
        with:
          creds: ${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}

      - name: validate template
        uses: Azure/cli@v1.0.0
        with:
          inlineScript: 'az deployment group validate --resource-group ${{ env.AZURE_RESOURCE_GROUP}} --template-file ${{ env.TEMPLATE_FILE }} --parameters ${{ env.PARAMETERS_FILE }}'

      - name: what-if
        uses: Azure/cli@v1.0.0
        with:
          inlineScript: 'az deployment group what-if --resource-group ${{ env.AZURE_RESOURCE_GROUP}} --template-file ${{ env.TEMPLATE_FILE }} --parameters ${{ env.PARAMETERS_FILE }}'
Enter fullscreen mode Exit fullscreen mode

Notice the reference to our service principal secrets (${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}) when login to the Azure CLI.

The continuous deployment job should behave a little bit different. First, it has only to run if the CI job was successful and if we're on the main branch. Second, there's no need to validate the template and rerun the what-if command. We can simply deploy the template:

jobs:
  CI:
    ...

  CD:
    needs: [CI]
    if: success() && (github.ref == 'refs/heads/main')

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: login to Azure
        uses: Azure/login@v1
        with:
          creds: ${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}

      - name: deploy
        uses: Azure/cli@v1.0.0
        with:
          inlineScript: 'az deployment group create --resource-group ${{ env.AZURE_RESOURCE_GROUP}} --template-file ${{ env.TEMPLATE_FILE }} --parameters ${{ env.PARAMETERS_FILE }}'
Enter fullscreen mode Exit fullscreen mode

That's it! If we now commit our workflow file, the actions get triggered. Since we're committing directly to the main branch, the CD action runs and executes our ARM template deployment. To see this in action, we can switch to GitHub and click on the Actions menu:
GitHub Action in progress
GitHub Action CI

To verify that all Azure resources have been created successfully, we can run the following Azure CLI command:

az resource list --query "[?resourceGroup=='rg-dev-blog'].{ name: name, resourceType: type }"
Enter fullscreen mode Exit fullscreen mode

The command returns a list of our deployed resources:

[
  {
    "name": "appi-dev-blog",
    "resourceType": "Microsoft.Insights/components"
  },
  {
    "name": "plan-dev-blog",
    "resourceType": "Microsoft.Web/serverFarms"
  },
  {
    "name": "app-dev-blog",
    "resourceType": "Microsoft.Web/sites"
  }
]
Enter fullscreen mode Exit fullscreen mode

GitHub actions for web

One thing left to complete the CI/CD pipeline for our Azure project: a GitHub workflow file for the web application. So let's open the empty web.yml file and define our action. Again, we first start with the name and the trigger:

name: Web

on:
  push:
    paths:
      - 'src/web/**'
      - '.github/workflows/web.yml'
    branches:
      - '**'
Enter fullscreen mode Exit fullscreen mode

Like in the workflow for our Azure infrastructure, this workflow should only get triggered by changes at the web application or the workflow file itself. We continue with some environment variables:

...

env:
  BUILD_CONFIGURATION: Release
  PUBLISH_OUTPUT: web_publish_output
  AZURE_WEBAPP_NAME: app-dev-blog
  WEB_PROJECT: ./src/web/HelloWorld.csproj
Enter fullscreen mode Exit fullscreen mode

The CI job will actually be quite simple:

  1. Check out source code
  2. Set up .NET Core 5
  3. Compile the project

Of course, if we had some unit tests, we would also execute them here:

...

jobs:
  CI:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 5.0.100

    - name: build web
      run: dotnet build ${{ env.WEB_PROJECT }}
Enter fullscreen mode Exit fullscreen mode

As in our previous workflow, the CD job should only run if the CI job was successful and if we're on the main branch. Again we check out the source code and set up .NET Core 5. After that, we use the dotnet CLI to publish the web application and deploy it via the Azure CLI:

jobs:
  CI:
    ...

  CD:
    needs: [CI]
    if: success() && (github.ref == 'refs/heads/main')

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Setup .NET Core
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: 5.0.100

      - name: publish web
        run: dotnet publish ${{ env.WEB_PROJECT }} -c ${{ env.BUILD_CONFIGURATION }} -o ${{ env.PUBLISH_OUTPUT }}

      - name: login to azure
        uses: Azure/login@v1
        with:
          creds: ${{ secrets.AZURE_DEPLOYMENT_CREDENTIALS }}

      - name: deploy to azure
        uses: Azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          package: './${{ env.PUBLISH_OUTPUT }}'
Enter fullscreen mode Exit fullscreen mode

After a commit to the main branch, the CI and CD jobs ran successfully, and the web application got deployed:
GitHub action web
Web application

Conclusion

As we saw in this blog post, it's not that hard to create a simple CI/CD pipeline. Of course, in a real-world project, the pipeline's and ARM templates' size and complexity will grow. That's exactly why it's important to start with them right at the beginning of the project. It's so much harder to set up a CI/CD pipeline when a project already got to a larger size and complexity.

You can find the complete code samples in this GitHub repository.

Top comments (2)

Collapse
 
nicojdejong profile image
nicojdejong •

Great write up! I was looking at Project Biceps a bit this morning - will keep that extension installed over the arm one.. pretty sure you already know about Biceps but still.

Collapse
 
manuelsidler profile image
Manuel Sidler •

Thanks, Nicoj! If you haven't seen yet, there are already two GitHub actions in the marketplace for the Bicep CLI:

Read next

developedbyviv profile image

I Passed the GitHub Foundations Certification: My Honest Experience

Vivek Singh -

omiossec profile image

Create a GitHub pipeline to test, review, and deploy a Bicep template.

Olivier Miossec -

brunobrito profile image

2024's Git Survey is Live!

Bruno Brito -

sanket00123 profile image

Behind the Scenes: Building a Dynamic Data Ingestion Pipeline with Azure Data Factory

SANKET PATIL -