Azure DevOps Pipelines with GitHub, Introduction

Azure DevOps is a set of tools to build and deliver product in an efficient way. It’s an online application with a board, Azure Board, for Scrum and Kaban, a repository, Azure Repos that use git and tfs, a Wiki, an artifact management, Azure Artefact, an exploratory test tool, Azure Test plan and a CI/CD pipeline, Azure DevOps Pipelines.
If you want more information about Azure DevOps, you can read this introduction

How can we start a Pipeline to test and build code from a GitHub repository with Azure DevOps Pipeline?
There are few steps:

• You need a GitHub account
• You need also an Azure DevOps account (you can, also, login to Azure DevOps with your GitHub account).

Let’s start.

For me, it’s better to create your account and your first Azure DevOps prject in a first time. You will have more control. Simply go to https://dev.azure.com

You can create an account or simply use your Github account.

In GitHub, we need to add the Azure DevOps extension; Simply click on "Marketplace" and search for Azure Pipeline.

Click on Set Up a plan

Select the Free plan and click on "Install it for free". Note that for public project, you get 10 parallel job and unlimited minutes per month.

Click to install to finish

You can, also, do the same with the Azure Board application.

The Azure DevOps project is not automatically public. If you want the 10 job and/or the unlimited build minutes you need to make sure your project visibility is set to public.

We can now create a build pipeline. There are two sorts of pipeline in Azure DevOps, GUI pipeline and YAML based pipeline.

A pipeline is a collection of tasks running on a specialized computer created only for this and that will be destroyed at the end. In Azure DevOps this computer is an agent. This agent came with tools to run build tasks, compiler (Go, .net, Visual Studio, ... ), runtime and SDK (.net, ...), tools (Git, Chocolatey, Nuget, NPM, …), script engines (PowerShell, Python, …), drivers.

There are several online agents:

• Windows 2019 with VS 2019
• Windows 2016 with VS 2017
• Windows 2012 R2 with VS 2015
• Windows Server 1803 Container image
• Mac OS 10.14
• Mac OS 10.13
• Ubuntu 16.04 lts

This computer run on Azure on a DS2 v2 VM. They are isolated and deleted at the end of the pipeline.
You can check this page to have more information about these agents.

For GUI Pipelines

In Azure DevOps go to Pipeline and click on New Pipeline

Click on "Use the classic editor"

In Select source, select GitHub, if you cannot see a connection to your GitHub repos you can still create one here.

Select your repository and the branch

Then select an empty job

We have, now, an integration pipeline. If you look on the right you can change the name of the pipeline and the default Agent. You can select:

Select the agent that fit your project, for the demonstration I will use "Hosted Windows 2019 with VS2019"

Click on "Save & queue" and select "Save" to save the modification (If you click on "Save & Queue" it will trigger the Pipeline and it will appear in the build history).

On the right, you can see the pipeline’s first task, "Get Sources". The source code from the branch will be copied on the agent each time the pipeline start.

You can control the behavior by clicking on it. Here you can tag the sources or you can checkout sub module.

You may need to use some value during the build. It can be a build number, a path, a module name or it can also be some secret values like passwords, keys, …. This is the purpose of the "variables" tab.

You can add some variables here, but the best way to that is to create a variable group and link it here.

To add a variable group, you need to go to Library in Pipeline Menu on the Left. Click on "+ Variable group" too add a new group. Give a name to the new variable group and click on "+ Add" to add a new variable and its value. 
Notice the padlock icon on the right. If you click on it the value of the variable is masked. The value will also be masked in the console or in the log. It’s not a security feature as a script can easily access to the value and every admin too. 

If you need to manage secrets with a pipeline you can enable a link to an Azure Subscription and select a Key Vault. Azure DevOps will act as a proxy to the key vault.
You can assess these variables in the agent host via environment variables during the build.

To create a build process, we need to add tasks to the agent job. Click on the + next to the Agent Job name and select one or more tasks. It can be a script task, a test, a compilation task … you can also browse the marketplace to find a task that fit your project.

Tasks generally include code compilation, unit and acceptance testing, packaging and other related tasks.

By default, if a task fails, it stops the process and report an error, but you can control this behavior by selecting the task and go to the "Control Option" on the left pane. You can use the "Continue on error" option to prevent the task to stop the process if it fails. It can be useful in a situation where running tests and publishing them are in two tasks.

By default, the pipeline is not trigger by any commit to the GitHub repos. We need to configure how the pipeline will react on a commit or a pull request.

The "Triggers" tab is where we can enable or disable continuous integration. You have two major options.

• Continuous Integration where every commit in the branch trigger the build process
• Pull Request Validation where every PR trigger the build process. You will need to select at least one branch.

Be aware, GitHub allow Pull Request from forked repos. Secrets associate with the build will not be accessible for the forked PR and may cause build error. You can enable these secrets with the “Make secrets available to build forks” but it may cause some security issue.

For YAML based Pipelines

The process to create a YAML based pipeline are not very different.

Instead of clicking on "Use the classic editor", you just need to click on GitHub. Select your GitHub repos and you will have to select an option, use a starter yaml file, a yaml file created in your repos by Azure DevOps or select an existing one.

For the first one, you will notice a new commit named "Set up CI with Azure Pipelines" and a new file azure-pipelines.yml in your GitHib repos. Most of the pipeline configuration is in this file.

The yaml file is a hierarchy

x Pipeline
x Stages

x Jobs
x Steps
x tasks

You can have the complete reference here.

But you can start with a more minimalist version.

Let start with a trigger, as in the GUI version you can control how the pipeline is triggered.

trigger: 
- master 

This will trigger the build pipeline every for every new commit in the master branch

In the same way we can enable pipeline for pull request for the same branch.

pr: 
- master 

We can also import a variables group from the library

variables: 
-  group: VarGroupName 

It's also possible to define hardcoded variables

variables: 
   modulename: MyModule 
   DefaultVersion: 0.0.1 

After that, we can setup the Agent host version

Pool: 
  vmImage: windows-2019

As for the GUI version there is a limited list of agents

And finally, we can add tasks we need under a Step

steps: 

- task: PowerShell@2 

  displayName: BuildScript 

  inputs: 

    targetType: filePath 

    filePath: '$(System.DefaultWorkingDirectory)\azdo-build.ps1' 

Notice the use of system variable here

As in the GUI version, you can add one or more task and you can control how the task behave in case of an error .

continueOnError: true 

Now that you see how we can setup a pipeline by using a single file you can see that yaml based pipeline are less easy but much more powerful than the GUI version.

A last thing about pipelines with Azure DevOps before the conclusion. How can advert to everyone the state of the build?

We can do it with badge. Badges are dynamic images hosted on Azure DevOps. The image change based on the status of the build, not yet run, success, fail. You can put in the README.md file of a GitHub project. To get the badge for the pipeline simple click on the 3 points on the left.

Click on "Status badge"

You just to copy and paste the Markdown into you README.md file.

This is just an introduction on how to use Azure DevOps Pipelines with GitHub. As you see YAML is the most powerful tool to create your CI. You can also create a CD pipeline with Azure DevOps to deliver the product.

If you can speak French you can view this video I made for the French PowerShell User Group in Paris about Build and Pipelines (but in French)