Adding PHPUnit Test Log and Coverage to GitLab CI/CD Pipeline

In this tutorial we are going to setup a GitLab CI/CD job that can run your PHPUnit test suite and extract a testing report plus the overall coverage. First, we need to setup our server and install the required tools. DigitalOcean has many useful tutorials and we are going to use some of them for the initial setup.

Prerequisites

• An Ubuntu 20.04 server setup by following this tutorial
• Install Docker following this tutorial
• Finally, installing Docker Compose by following this tutorial

Server Setup

Now that the server’s initial setup is complete, our next steps will be: installing GitLab runner, register a runner for your project and create a user for deployment.

Installing GitLab Runner

1- Adding the official gitlab-runner repo and inspecting the security of the installing script

   curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh > script.deb.sh
   less script.deb.sh

2- Running the installer

   sudo bash script.deb.sh

3- Install gitlab-runner service

   sudo apt install gitlab-runner

4- Check the service status

   systemctl status gitlab-runner

The output should be something like this

   Output
   ● gitlab-runner.service - GitLab Runner
      Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled; vendor preset: enabled)
      Active: active (running) since Mon 2020-06-01 09:01:49 UTC; 4s ago
    Main PID: 16653 (gitlab-runner)
       Tasks: 6 (limit: 1152)
      CGroup: /system.slice/gitlab-runner.service
              └─16653 /usr/lib/gitlab-runner/gitlab-runner run --working-directory /home/gitlab-runner --config /etc/gitla

Register a Runner for Your GitLab Project

1- In your GitLab project, navigate to Settings > CI/CD > Runners.

2- In the Specific Runners section, you’ll find the registration token and the GitLab URL. Copy both as we’ll need them for the next command.

3- Now in the terminal, register the runner by running

   sudo gitlab-runner register -n --url https://your_gitlab.com --registration-token project_token --executor shell --description "Staging Shell Runner" --tag-list deployment

This seems like a lot of options but they’re pretty easy to understand:

• -n executes the register command non-interactively.
• --url is the GitLab URL from step 2.
• --registration-token is the token you copied from the runners page in step 2.
• --executor is the executor type, we are using shell executer here which is a simple executor that you use to execute builds locally on the machine where GitLab Runner is installed.
• --description is the runner’s description, which will show up in GitLab.
• --tag-list is a list of tags assigned to the runner. Tags can be used in a pipeline configuration to select specific runners for a CI/CD job. The deployment tag will allow you to refer to this specific runner to execute the deployment job.

4- The output of the above command will return an output like this:

   Output
   Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

And the runner will show up in your project’s Settings > CI/CD > Runners

Create Deployment User

1- Create a new user

   sudo adduser deployer

2- Add user to docker group to permit deployer to execute the docker command, which is required to perform the deployment

   sudo usermod -aG docker deployer

3- Create SSH key for deployer

Switch to the deployer user

   su deployer

Then generate a 4096-bit SSH key

   ssh-keygen -b 4096

> ⚠ Do not choose a password for your key or you will have to enter it with each job. This is not possible since our runner is non-interactive

Add the new key to the authorized keys

```bash

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

4- Storing the private key in a GitLab CI/CD variable

   First, get the private key be executing

   ```bash


   cat ~/.ssh/id_rsa

Copy the output and navigate to Settings > CI / CD > Variables and click Add Variable

fill the variable form like this

• Key: ID_RSA
• Value: Paste your SSH private key from your clipboard (including a line break at the end).
• Type: File
• Environment Scope: All (default)
• Protect variable: Checked
• Mask variable: Unchecked

Create another variable for your server IP address

• Key: SERVER_IP
• Value: your_server_IP
• Type: Variable
• Environment scope: All (default)
• Protect variable: Checked
• Mask variable: Checked

And the last variable is for the user

• Key: SERVER_USER
• Value: deployer
• Type: Variable
• Environment scope: All (default)
• Protect variable: Checked
• Mask variable: Checked

Configuring .gitlab-ci.yml File

Next, we will be prepare the docker file for testing environment, the application service in docker compose file and finally PHPUnit job in .gitlab-ci.yml

Application Docker file

In your app docker file we will need to install XDebug extension to collect the testing coverage report and create two new files: phpunit-report.xml and phpunit-coverage.xml for testing report and testing coverage.

We will create the file in ./docker/testingApp.dockerfile with minimal configuration. It should be something like this.

FROM php:8.1.3-fpm-alpine

WORKDIR /var/www/

# Install alpine packages
RUN apk add --no-cache --update # Add your packages

# Install php extensions
RUN docker-php-ext-install # Add the PHP extension you need

# Install XDebug
RUN pecl install xdebug \
    && docker-php-ext-enable xdebug 

# Copy existing application directory contents
COPY --chown=www-data:www-data . .

RUN touch phpunit-report.xml phpunit-coverage.xml
RUN chmod 777 phpunit-report.xml phpunit-coverage.xml

USER www-data

Application Service

The app service in ./docker/docker-compose-testing.yml will have two volumes: one for the tests directory and the other is ./docker/php/conf.d/xdebug.ini that will contain the basic XDebug configuration.

The app service should look like this:

version: "3"
services:
  app:
    container_name: "app-testing"
    build:
      context: ../
      dockerfile: ./docker/testingApp.dockerfile
    volumes:
      - ./php/conf.d/xdebug.ini:/usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
    working_dir: /var/www

And xdebug.ini is quite minimal, we just set enable the coverage mode but you can enable multiple modes at the same time. Learn more about XDebug modes from the documentation

zend_extension=xdebug

[xdebug]
xdebug.mode=coverage

PHPUnit Job

In .gitlab-ci.yml we will create a job for PHPUnit that will have the deployment tag so that it uses our runner and extract the required artifacts for GitLab to display test reports.

The job will be

phpunit:
  stage: test
  tags:
    - deployment
  before_script:
    - docker-compose -p my-project -f docker/docker-compose-testing.yml build
    - docker-compose -p my-project -f docker/docker-compose-testing.yml up -d
    - docker exec $CONTAINER_NAME php artisan migrate
    - docker exec $CONTAINER_NAME php artisan db:seed
  script:
    - docker exec -t $CONTAINER_NAME vendor/bin/phpunit --do-not-cache-result --log-junit phpunit-report.xml --coverage-cobertura phpunit-coverage.xml --coverage-text --colors=never
    - docker cp $CONTAINER_NAME:/var/www/phpunit-report.xml ./
    - docker cp $CONTAINER_NAME:/var/www/phpunit-coverage.xml ./
  after_script:
    - docker-compose -p my-project -f docker/docker-compose-testing.yml down
  artifacts:
    when: always
    reports:
      junit: phpunit-report.xml
      coverage_report:
        coverage_format: cobertura
        path: phpunit-coverage.xml
  coverage: '/^\s*Lines:\s*\d+.\d+\%/'
  only:
    - merge_requests
    - main
    - develop

You should notice that we build our containers using docker-compose-testing.yml, run the DB migrations and seeder then run our test suit with a few options:

docker exec -t $CONTAINER_NAME vendor/bin/phpunit --do-not-cache-result --log-junit phpunit-report.xml --coverage-cobertura phpunit-coverage.xml --coverage-text --colors=never

• --do-not-cache-result to disable test result caching.
• --log-junit phpunit-report.xml specify the test log format and the output file.
• --coverage-cobertura phpunit-coverage.xml specify the coverage full report format and the output file.
• --coverage-text generate code coverage report in text format.
• --colors=never disable colors in the output to make it easier to extract the coverage percentage from the command output.

Then we copy the generated reports from inside the container to the pipeline namespace to be used as job artifacts

docker cp $CONTAINER_NAME:/var/www/phpunit-report.xml ./
docker cp $CONTAINER_NAME:/var/www/phpunit-coverage.xml ./

The job will have two artifacts of type reports the first is the a junit for test log and a cobertura report for the test coverage.

artifacts:
    when: always
    reports:
      junit: phpunit-report.xml
      coverage_report:
        coverage_format: cobertura
        path: phpunit-coverage.xml

ℹ The report formats and files’ extensions are specified by GitLab

Project Configuration

At this point we have created a job for PHPUnit that we run on our Staging Shell Runner that we registered on the staging server. GitLab will be able to display a test report that include some helpful information like the total execution time of the test suite and the success rate. It will also show the test coverage on the jobs and in the merge request overview.

To go one step further, we can track the test coverage history and add coverage badge to the project

Test Coverage History Using Project Settings (Deprecated in GitLab 14.9)

Navigate to Settings > CI/CD > General Pipelines and in the Test Coverage Parsing field add the same regular expression that we used in the job ^\s*Lines:\s*\d+.\d+\%

Test Coverage Badge

Navigate to Settings > General Settings > Badges and click Add Badge and fill the form as follows:

• Name: PHPUnit Coverage
• Link: https://gitlab.com/[PROJECT_PATH]/-/commits/[BRANCH_NAME]
• Badge Image URL: https://gitlab.com/[PROJECT_PATH]/badges/[BRANCH_NAME]/coverage.svg

Result

• You can see a test report in your pipeline

• Navigate to Analytics > Repository > Code Coverage Statistics to see the history of test coverage

• The effect of each merge request on the test coverage can be found in the merge request overview page.