Gitlab Integration with Sonarqube: Automate your quality gates

TL/DR: A new parameter made this post for a lot of use cases obsolete. You can activate sonar.qualitygate.wait to have a built in waiting mechanism which exits the plugin with a non-zero return code on a failing quality gate. More information at: https://docs.sonarqube.org/latest/analysis/gitlab-cicd/

I accidentally killed the first release of this article so here its up again. Any improvement suggestions very welcome.

This article was originally posted on my companies blog at viesure.io/blog.

Last week I took the task of our sprint to create a build breaker step for Sonar in our GitLab deployment pipeline.
To clarify this term: We wanted each build to break if the quality gate of Sonar fails.
Since this was not straightforward, I wanted to share my result.
I am using the API of the SonarQube server to get the information about the status of the quality gate.

Preconditions to use the script

A Sonar API Token that has sufficient rights to access the data of the analysis. If you don't know where to get the token look it up in the documentation
A sonar runner needs to be executed before, to get a ceTaskId for the requests. For maven the sonar:sonar runner creates a file at target/sonar/report-task.txt.

#Example content of the report-task.txt
projectKey=io.viesure:project
serverUrl=https://sonar.server.url
serverVersion=7.7.0.23042
branch=master
dashboardUrl=https://sonar.server.url/dashboard?id=io.viesure%3Aproject&branch=master
ceTaskId=AWy0noy7Aqf9WEmmjKgw
ceTaskUrl=https://sonar.server.url/api/ce/task?id=AWy0noy7Aqf9WEmmjKgw

A detailed explanation of the script

In this section, I will explain the parts of the script step by step.
TL;DR at the end is a link to the full script on GitHub.

Semi-static parameters

Since we are using Sonar with Maven in a CI/CD pipeline, I hard link the source for the result. Depending on your runner, this might differ. A better practice would also be to also extract the sonar server property to the CI environment variables, to reduce the nonfunctional information within the code.

#!/usr/bin/env sh

SONAR_RESULT=${2:-"target/sonar/report-task.txt"}
SONAR_SERVER=${1:-"https://our.sonar.server"}

Sonar API Token

We are checking if the environment has set the sonar API token. If this variable is not available, we terminate the script with a proper error message.

if [ -z $SONAR_API_TOKEN ]
then
  echo "Sonar API Token not set."
  exit 1
fi

Extract ceTaskId from Sonar result and validate access rights

First, we are checking if the sonar result file exists.

if [ ! -f $SONAR_RESULT ]
then
  echo "Sonar result does not exist"
  exit 1
fi

Afterwards, we use sed to extract the task id for further processing. The task id is a unique hash that refers to a sonar runner result that is processed by the server.

CE_TASK_ID=`sed -n 's/ceTaskId=\(.*\)/\1/p' < $SONAR_RESULT`

if [ -z $CE_TASK_ID ]
then
  echo "ceTaskId is not set from sonar build."
  exit 1
fi

Finally, we are checking if our API Token has proper access rights to get information about the processing state of the found task id.

HTTP_STATUS=$(curl -s -o /dev/null -w '%{http_code}' -u $SONAR_API_TOKEN: $SONAR_SERVER/api/ce/task\?id\=$CE_TASK_ID)

if [  "$HTTP_STATUS" -ne 200 ]
then
  echo "Sonar API Token has no access rights."
  exit 1
fi

Find analysisId for a taskId

After processing a project with a sonar runner the SonarQube server processes the result to generate an analysis report. The task.analysisId from the response JSON is available after the process has finished. For this reason, I created a simple time loop to evaluate the analysisId from the response. Jq returns null if the property that is filtered is not part of the JSON.

ANALYSIS_ID=$(curl -XGET -s -u $SONAR_API_TOKEN: $SONAR_SERVER/api/ce/task\?id\=$CE_TASK_ID | jq -r .task.analysisId)
I=1
TIMEOUT=0
while [ $ANALYSIS_ID = "null" ]
do
  if [ "$TIMEOUT" -gt 30 ]
  then
    echo "Timeout of " + $TIMEOUT + " seconds exceeded for getting ANALYSIS_ID"
    exit 1
  fi
  sleep $I
  TIMEOUT=$((TIMEOUT+I))
  I=$((I+1))
  ANALYSIS_ID=$(curl -XGET -s -u $SONAR_API_TOKEN: $SONAR_SERVER/api/ce/task\?id\=$CE_TASK_ID | jq -r .task.analysisId)
done

Get the quality gate information

The quality gate API endpoint returns a JSON that contains all parts of the quality analysis from Sonar. The projectStatus.status is representing the state of the quality gate and was our desired health metric. If you want to have a different setup, adapt the jq processing to your needs and find your breaking parameter.

STATUS=$(curl -XGET -s -u $SONAR_API_TOKEN: $SONAR_SERVER/api/qualitygates/project_status?analysisId=$ANALYSIS_ID | jq -r .projectStatus.status)

if [ $STATUS = "ERROR" ]
then
  echo "Qualitygate failed."
  exit 1
fi

echo "Sonar Qualitygate is OK."
exit 0

Executing the script in the pipeline

First, execute the script after the sonar execution.

#!/usr/bin/env sh

export MAVEN_OPTS="-Dmaven.repo.local=maven.repository -Dsonar.branch.name=$CI_COMMIT_REF_NAME"

./mvnw sonar:sonar
. ${SCRIPT_DIR}/sonar_break_build.sh

Then define a job that executes sonar and the breaker.

sonar:
    stage: sonar
    tags: [java11]
    dependencies:
        - mvn-build
    script:
        - ${SCRIPT_DIR}/mvn_sonarqube.sh
    only:
        - branches
    cache:
        paths:
            - maven.repository/

More detailed information about the script can is available at our blog post about optimizing our build times with gitlab on aws

Conclusion

Our goal is to deliver a good maintainable code as fast as possible.
To achieve this fast feedback is crucial and breaking our build, on the first commit that introduces new security issues, bugs and code smells, is game-changing for this goal.

Some colleagues might be annoyed at first, that their code is instantly analyzed or they have to fix stuff that does not pass the quality gate right away. On the upside code that does not fit the quality standard won't ever be merged to develop or master.
Also, you now need to interact with Sonar more often, but this can also be an advantage to shape your ruleset to the degree that everybody can live with it on a daily basis.
Another question on fine-tuning that came up was if we should break the build on Sonar or mark it as unstable to minimize the disruption of the pipeline, but still get the feedback from Sonar.

I hope you enjoyed the read.
Additional, I would love some feedback on the shell script to make it as robust as possible, since its part of a deploy pipeline.
What's your opinion of breaking your pipeline with Sonar?

viesure/blog-sonar-build-breaker