In this tutorial we are going to setup a GitLab CI/CD job that can run your PHPUnit
test suite and extract a testing report plus the overall coverage. First, we need to setup our server and install the required tools. DigitalOcean
has many useful tutorials and we are going to use some of them for the initial setup.
Prerequisites
- An
Ubuntu 20.04
server setup by following this tutorial - Install
Docker
following this tutorial - Finally, installing
Docker Compose
by following this tutorial
Server Setup
Now that the server’s initial setup is complete, our next steps will be: installing GitLab runner, register a runner for your project and create a user for deployment.
Installing GitLab Runner
1- Adding the official gitlab-runner
repo and inspecting the security of the installing script
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh > script.deb.sh
less script.deb.sh
2- Running the installer
sudo bash script.deb.sh
3- Install gitlab-runner
service
sudo apt install gitlab-runner
4- Check the service status
systemctl status gitlab-runner
The output should be something like this
Output
● gitlab-runner.service - GitLab Runner
Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-06-01 09:01:49 UTC; 4s ago
Main PID: 16653 (gitlab-runner)
Tasks: 6 (limit: 1152)
CGroup: /system.slice/gitlab-runner.service
└─16653 /usr/lib/gitlab-runner/gitlab-runner run --working-directory /home/gitlab-runner --config /etc/gitla
Register a Runner for Your GitLab Project
1- In your GitLab project, navigate to Settings > CI/CD > Runners.
2- In the Specific Runners section, you’ll find the registration token and the GitLab URL. Copy both as we’ll need them for the next command.
3- Now in the terminal, register the runner by running
sudo gitlab-runner register -n --url https://your_gitlab.com --registration-token project_token --executor shell --description "Staging Shell Runner" --tag-list deployment
This seems like a lot of options but they’re pretty easy to understand:
-
-n
executes the register command non-interactively. -
--url
is the GitLab URL from step 2. -
--registration-token
is the token you copied from the runners page in step 2. -
--executor
is the executor type, we are usingshell
executer here which is a simple executor that you use to execute builds locally on the machine where GitLab Runner is installed. -
--description
is the runner’s description, which will show up in GitLab. -
--tag-list
is a list of tags assigned to the runner. Tags can be used in a pipeline configuration to select specific runners for a CI/CD job. Thedeployment
tag will allow you to refer to this specific runner to execute the deployment job.
4- The output of the above command will return an output like this:
Output
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
And the runner will show up in your project’s Settings > CI/CD > Runners
Create Deployment User
1- Create a new user
sudo adduser deployer
2- Add user to docker group to permit deployer
to execute the docker
command, which is required to perform the deployment
sudo usermod -aG docker deployer
3- Create SSH key for deployer
Switch to the deployer
user
su deployer
Then generate a 4096-bit SSH key
ssh-keygen -b 4096
> ⚠ Do not choose a password for your key or you will have to enter it with each job. This is not possible since our runner is non-interactive
Add the new key to the authorized keys
```bash
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
4- Storing the private key in a GitLab CI/CD variable
First, get the private key be executing
```bash
cat ~/.ssh/id_rsa
Copy the output and navigate to Settings > CI / CD > Variables and click Add Variable
fill the variable form like this
- Key:
ID_RSA
- Value: Paste your SSH private key from your clipboard (including a line break at the end).
- Type: File
- Environment Scope: All (default)
- Protect variable: Checked
- Mask variable: Unchecked
Create another variable for your server IP address
- Key:
SERVER_IP
- Value:
your_server_IP
- Type: Variable
- Environment scope: All (default)
- Protect variable: Checked
- Mask variable: Checked
And the last variable is for the user
- Key:
SERVER_USER
- Value:
deployer
- Type: Variable
- Environment scope: All (default)
- Protect variable: Checked
- Mask variable: Checked
Configuring .gitlab-ci.yml
File
Next, we will be prepare the docker file for testing environment, the application service in docker compose file and finally PHPUnit job in .gitlab-ci.yml
Application Docker file
In your app docker file we will need to install XDebug extension to collect the testing coverage report and create two new files: phpunit-report.xml
and phpunit-coverage.xml
for testing report and testing coverage.
We will create the file in ./docker/testingApp.dockerfile
with minimal configuration. It should be something like this.
FROM php:8.1.3-fpm-alpine
WORKDIR /var/www/
# Install alpine packages
RUN apk add --no-cache --update # Add your packages
# Install php extensions
RUN docker-php-ext-install # Add the PHP extension you need
# Install XDebug
RUN pecl install xdebug \
&& docker-php-ext-enable xdebug
# Copy existing application directory contents
COPY --chown=www-data:www-data . .
RUN touch phpunit-report.xml phpunit-coverage.xml
RUN chmod 777 phpunit-report.xml phpunit-coverage.xml
USER www-data
Application Service
The app
service in ./docker/docker-compose-testing.yml
will have two volumes: one for the tests
directory and the other is ./docker/php/conf.d/xdebug.ini
that will contain the basic XDebug configuration.
The app service should look like this:
version: "3"
services:
app:
container_name: "app-testing"
build:
context: ../
dockerfile: ./docker/testingApp.dockerfile
volumes:
- ./php/conf.d/xdebug.ini:/usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
working_dir: /var/www
And xdebug.ini is quite minimal, we just set enable the coverage
mode but you can enable multiple modes at the same time. Learn more about XDebug modes from the documentation
zend_extension=xdebug
[xdebug]
xdebug.mode=coverage
PHPUnit Job
In .gitlab-ci.yml
we will create a job for PHPUnit
that will have the deployment
tag so that it uses our runner and extract the required artifacts for GitLab to display test reports.
The job will be
phpunit:
stage: test
tags:
- deployment
before_script:
- docker-compose -p my-project -f docker/docker-compose-testing.yml build
- docker-compose -p my-project -f docker/docker-compose-testing.yml up -d
- docker exec $CONTAINER_NAME php artisan migrate
- docker exec $CONTAINER_NAME php artisan db:seed
script:
- docker exec -t $CONTAINER_NAME vendor/bin/phpunit --do-not-cache-result --log-junit phpunit-report.xml --coverage-cobertura phpunit-coverage.xml --coverage-text --colors=never
- docker cp $CONTAINER_NAME:/var/www/phpunit-report.xml ./
- docker cp $CONTAINER_NAME:/var/www/phpunit-coverage.xml ./
after_script:
- docker-compose -p my-project -f docker/docker-compose-testing.yml down
artifacts:
when: always
reports:
junit: phpunit-report.xml
coverage_report:
coverage_format: cobertura
path: phpunit-coverage.xml
coverage: '/^\s*Lines:\s*\d+.\d+\%/'
only:
- merge_requests
- main
- develop
You should notice that we build our containers using docker-compose-testing.yml
, run the DB migrations and seeder then run our test suit with a few options:
docker exec -t $CONTAINER_NAME vendor/bin/phpunit --do-not-cache-result --log-junit phpunit-report.xml --coverage-cobertura phpunit-coverage.xml --coverage-text --colors=never
-
--do-not-cache-result
to disable test result caching. -
--log-junit phpunit-report.xml
specify the test log format and the output file. -
--coverage-cobertura phpunit-coverage.xml
specify the coverage full report format and the output file. -
--coverage-text
generate code coverage report in text format. -
--colors=never
disable colors in the output to make it easier to extract the coverage percentage from the command output.
Then we copy the generated reports from inside the container to the pipeline namespace to be used as job artifacts
docker cp $CONTAINER_NAME:/var/www/phpunit-report.xml ./
docker cp $CONTAINER_NAME:/var/www/phpunit-coverage.xml ./
The job will have two artifacts of type reports
the first is the a junit
for test log and a cobertura
report for the test coverage.
artifacts:
when: always
reports:
junit: phpunit-report.xml
coverage_report:
coverage_format: cobertura
path: phpunit-coverage.xml
ℹ The report formats and files’ extensions are specified by GitLab
Project Configuration
At this point we have created a job for PHPUnit that we run on our Staging Shell Runner
that we registered on the staging server. GitLab will be able to display a test report that include some helpful information like the total execution time of the test suite and the success rate. It will also show the test coverage on the jobs and in the merge request overview.
To go one step further, we can track the test coverage history and add coverage badge to the project
Test Coverage History Using Project Settings (Deprecated in GitLab 14.9)
Navigate to Settings > CI/CD > General Pipelines and in the Test Coverage Parsing field add the same regular expression that we used in the job ^\s*Lines:\s*\d+.\d+\%
Test Coverage Badge
Navigate to Settings > General Settings > Badges and click Add Badge and fill the form as follows:
- Name: PHPUnit Coverage
- Link:
https://gitlab.com/[PROJECT_PATH]/-/commits/[BRANCH_NAME]
- Badge Image URL:
https://gitlab.com/[PROJECT_PATH]/badges/[BRANCH_NAME]/coverage.svg
Result
- You can see a test report in your pipeline
- Navigate to Analytics > Repository > Code Coverage Statistics to see the history of test coverage
- The effect of each merge request on the test coverage can be found in the merge request overview page.
Top comments (4)
If you use Laravel, so accordingly with the command
php artisan test --coverage
the tests, then the regex is not correct. Instead just use this regex
coverage: '/^\s*Total\sCoverage\s\.+\s\d+\.\d+\s\%/'
this is not possible anymore in gitlab. you just have to use the coverage-property on the job (like described in the blog post!)
You're right. I've updated it thank you!
Just wondering how to set up gitlab to make every test coverage showing in the merge request overview page?