A Novice's Guide to Integrating Code Coverage Report with SonarQube and Gitlab Runner

Integrating SonarQube is crucial for identifying breaking changes and technical debt in large codebases. Automating unit tests and setting up a SonarQube job for code analysis can help with this. However, configuring the pipeline can be difficult, and available documentation may not be intuitive for those new to DevOps like myself. This guide aims to assist developers in setting up and testing CI/CD jobs on their own machine. One case example is to integrate code coverage report to SonarQube analysis in a gitlab job that could be tested locally.

1. Installing sonarqube

Visit here for a straight-forward SonarQube installation guide. Enterprise usually have a centralized SonarQube dashboard which you can connect to. However, for learning purposes we will install and run it on our machine instead.

2. Setup sonar config on the project's root

Go to the root of your project and create a new sonar-project.properties file.

# Project specification
sonar.projectKey=project-key-example
sonar.projectName=project-name-example
sonar.projectVersion=project-version-example

# Analysis specification
sonar.language=go
sonar.exclusions=**/mocks/**,**/vendor/**
sonar.qualitygate.wait=true

# Testing specification
sonar.go.coverage.reportPaths=./coverage.out
sonar.go.coverage.minimumCoverage=80
sonar.test.exclusions=**/*_test.go,**/mocks/**,**/vendor/**

Project specification will be necessary to identify your project among others that will share your SonarQube dashboard. So, make sure that the key is unique and the name is descriptive.

Analysis specification is important to tell sonar what framework it needs to use in running the analysis. The things they will analyze by default are: code smells, code duplication, and bugs. By default, coverage analysis won't be done unless we feed the report into it, which we'll do by the end of this article.

Specify the language you're using in sonar-language, in this case we'll use Go. Exclude the folders that won't be needing analysis, such as mock files or vendor files, and specify it under sonar.exclusions. You can also use sonar.inclusions to specify the folders that needs to be analyzed. But since by default we want everything we write to be put under scrutiny, only specifying exclusions would suffice. sonar.qualitygate.wait=true will tell the SonarQube analysis to wait for the completion of the quality gate evaluation before completing the analysis.

Quality gate could be another topic for another time, but to put simply, it is a set of threshold that would qualify your code to be production ready. This threshold could consist of percentage of code coverage, number of code smells, etc. Waiting for this evaluation may not be necessary at all, unless you want to make sure every deploy is production-ready.

Next is specifying the tests. sonar.go.coverage.reportPaths=./coverage.out tells sonar where it needs to read coverage report, with sonar.go.coverage.minimumCoverage=80 setting the minimum percent of coverage. You'd also want to list untested files, including test files themselves, under sonar.test.exclusions.

That's it! We're done with configuring sonar file.

3. Setup Gitlab CI jobs

Find these stages under the existing .gitlab-ci.yml in your project, or create a new one if it doesn't exist.

stages:
   - build
   - test
   - deploy

Usually, there are three main stages to CI: build, test, and deploy. But now, we not only want to do test as part of the stage, but also analyzing the codebase as a whole. You can add a new analyze stage after test, or you can put them both under one stage. In my case, I preferred grouping them together under analyze stage.

stages:
   - build
   - analyze
   - deploy

Next, we're going to create two jobs: unit-test and sonar-analysis.

unit-test:
  stage: analyze
  image: golang:1.18
  script:
    - go test -v -coverprofile=coverage.out -covermode=set ./...

sonar-analysis:
   stage: analyze
   image: sonarsource/sonar-scanner-cli:latest
   script:
     - sonar-scanner

See that both of these jobs belong under the same analyze stage, while also running on different docker images which would allow each jobs to have their own dependencies. After setting it up this way, both jobs will be running simultaneously, which won't allow us to integrate coverage report with sonar analysis. Instead, what we want is to run unit-test first, generate coverage report, and then run overall code analysis on top of the report.

To do this, we're going to add the artifacts.paths to save our generated coverage report. Then, tell sonar-analysis job to wait for unit-test by adding needs.job and then for it to download and use the artifact produced by unit-test by adding needs.artifacts.

unit-test:
  stage: analyze
  image: golang:1.18
  script:
    - go test -v -coverprofile=coverage.out -covermode=set ./...
  artifacts:
     paths:
       - coverage.out

sonar-analysis:
   stage: analyze
   image: sonarsource/sonar-scanner-cli:latest
   script:
     - sonar-scanner
   needs:
     - job: unit-test
       artifacts: true

You can add other configurations too, such as allow_failure: true so any failures won't block the rest of the stage. You can also specify the branches to run these by adding only.

   ...
   allow_failure: true
   only: 
     - develop
     - master
  ...

Also an important thing to note, this example does not cover everything. For example, you'll have to connect the sonar-analysis into your SonarQube dashboard, which we don't cover in this article.

4. Testing Gitlab CI jobs in local environment

Testing .gitlab-ci.yml changes was the trickier part, since it runs on Gitlab Runner and can't be tested with regular docker commands. In order to do this, we need to install Gitlab Runner on our machine, and then test the .gitlab-ci.yml file on top of it.

Gitlab Runner Installation

Install gitlab runner here, or install from linux repository

Setting up Gitlab Runner

• Make sure installation succeeded by running gitlab-runner help
• Register gitlab-runner

sudo gitlab-runner register

You will be prompted to input gitlab instance url and token, which can be found the Settings of your repo under CI/CD mennu. Expand Runners section and find the token under Specific runners

• Enter runner description and tags
• Choose executor, we'll use docker for our project. See here for other options
• For detailed instruction, see here

Run test

After successfully setting up runner, you can start the service and then go to the project's root directory. To test one specific job, you can run

gitlab-runner exec docker job_name

Do this to test the whole pipeline

gitlab-runner run

If your gitlab-ci.yml isn't detected, a config flag can be added to specify file's location

gitlab-runner run --config /path/to/project/.gitlab-ci.yml

That's it! You should be able to execute Gitlab CI now on your machine and see SonarQube analysis running in real time.

Image by vectorjuice on Freepik