There is a bad notion about you can’t make Code Coverage in .Net Core on Mac, because OpenCover tool only works on Windows.
But I had discovered a new NuGet package called Coverlet a cross-platform code coverage library for .NET Core. Then finally we can make code coverage using the same commands no matters if you are working on Mac, Linux or Windows.
Coverlet
Coverlet - Cross platform code coverage tool for .NET Core
It’s unbelievable easy to use, comparing with OpenCover. Add this NuGet package in your test project:
dotnet add package coverlet.msbuild
Now add some parameters when running the test project:
dotnet test /p:CollectCoverage=true
And that’s all. But if you want personalize it according your needs:
dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=\"opencover,lcov\" /p:CoverletOutput=../lcov
I need a code coverage report in opencover and lcov formats. The first one to be uploaded on Sonar Cloud and the second one to be used with Coverage Gutters Visual Studio Code extension.
Visual Studio Code Extensions
If Ash Ketchum were a programmer, surely would say: Visual Studio Code...
It can be extensible according to what you need. And it’s perfect because these two extensions give you unit test superpowers:
Coverage Gutter display coverage result with colors in your screen and you can activate or deactivate it. And Test Explorer gives you a visual explorer panel when you can run tests: all of them, a group in context or individual test. Even better lights up code lens style over each test and you can see his result.
Unit Testing Framework
You can found a lot of literature about unit test frameworks (xUnit, nUnit, and MSTest) in .Net, and really no matters what you choose, there are no significant differences between them.
Why I choose xUnit? because its part of .Net Foundations, I like his syntaxis and works like a charm with Test Explorer plugin.
There is a good explanation about xUnit with .Net Core. If you are making baby steeps I highly recommend reading it:
Unit testing C# code in .NET Core using dotnet test and xUnit
Sonar Cloud
I can’t imagine a present without using a tool like SonarQube in a project. I have created two scripts to run Sonar Scanner and upload the results including code coverage.
Install SonarScanner for MSBuild:
dotnet tool install --global dotnet-sonarscanner
Create a sonar login token on SonarCloud on Profile > Security > Generate Token. Save it inside sonar.txt file and add this file to .gitignore. Now you can use it locally and can’t be revealed to curious eyes.
Use this on Windows (Bat):
@echo off
set /p token=<sonar.txt
dotnet sonarscanner begin /k:"company:project" /n:"Project" /v:"#.#.#" /o:"companyname" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="%token%" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.coverage.exclusions="Project.Tests/**,**/*Tests.cs" /d:sonar.cs.opencover.reportsPaths="%cd%\lcov.opencover.xml"
dotnet restore
dotnet build
dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=\"opencover,lcov\" /p:CoverletOutput=../lcov
dotnet sonarscanner end /d:sonar.login="%token%"
Use this on Mac/Linux (bash) and remember give execution permissions:
#!/bin/bash
token="$(cat sonar.txt)"
dir="$(pwd)"
dotnet sonarscanner begin /k:"company:project" /n:"Project" /v:"#.#.#" /o:"company" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="${token}" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.cs.opencover.reportsPaths="${dir}/lcov.opencover.xml"
dotnet restore
dotnet build
dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=\"opencover,lcov\" /p:CoverletOutput=../lcov
dotnet sonarscanner end /d:sonar.login="${token}"
Change company , project , #.#.# and Project words with yours.
Analysis Parameters - SonarQube Documentation - Doc SonarQube
Bonus Track
Tasks
Another loved feature is Visual Studio Code is that you can automate and create tasks and can be easily launched.
That’s my configuration task.json file. I have automated build, test, publish, pack and sonar tasks.
{
"version": "2.0.0",
"tasks": [
{
"label": "build",
"command": "dotnet",
"type": "process",
"group": {
"kind": "build",
"isDefault": true
},
"args": [
"build",
"${workspaceFolder}/Project/Project.csproj"
],
"problemMatcher": "$msCompile"
},
{
"label": "build tests",
"command": "dotnet",
"type": "process",
"group": "build",
"args": [
"build",
"${workspaceFolder}/Project.Tests/Project.Tests.csproj"
],
"problemMatcher": "$msCompile"
},
{
"label": "test",
"command": "dotnet",
"type": "process",
"group": {
"kind": "test",
"isDefault": true
},
"args": [
"test",
"${workspaceFolder}/Project.Tests/Project.Tests.csproj",
"/p:CollectCoverage=true",
"/p:CoverletOutputFormat=\"opencover,lcov\"",
"/p:CoverletOutput=../lcov"
],
"problemMatcher": "$msCompile"
},
{
"label": "publish win",
"command": "dotnet",
"group": "none",
"args": [
"publish",
"${workspaceRoot}/Project/Project.csproj",
"-o",
"${workspaceRoot}/Library/win/",
"-c",
"release",
"-r",
"win10-x64"
],
"problemMatcher": "$msCompile"
},
{
"label": "publish mac",
"command": "dotnet",
"args": [
"publish",
"${workspaceRoot}/Project/Project.csproj",
"-o",
"${workspaceRoot}/Library/mac/",
"-c",
"release",
"-r",
"osx.10.12-x64"
],
"problemMatcher": "$msCompile"
},
{
"label": "pack",
"command": "dotnet",
"args": [
"pack",
"${workspaceRoot}/Project/Project.csproj",
"/p:NuspecFile=${workspaceRoot}/Project/Project.nuspec",
"-o",
"${workspaceRoot}/Package/",
"-c",
"release"
],
"problemMatcher": "$msCompile"
},
{
"label": "permissions",
"type": "shell",
"osx": {
"command": "chmod +x ${workspaceRoot}/sonar.sh"
},
"presentation": {
"reveal": "always",
"panel": "new"
},
"problemMatcher": []
},
{
"label": "sonar",
"type": "shell",
"windows": {
"command": "${workspaceRoot}\\sonar.bat"
},
"osx": {
"command": "${workspaceRoot}/sonar.sh"
},
"presentation": {
"reveal": "always",
"panel": "new"
},
"problemMatcher": []
}
]
}
Change company , project , #.#.# and Project words with yours.
Travis CI
Run SonarScanner on Travis it’s a little problematic. The trick is to export the path when it’s installed.
In order to keep secret the sonar key, create an environment variable on Travis as SONAR_KEY.
language: csharp
mono: none
dotnet: 3.1.201
solution: Project/Project.csproj
install:
- dotnet tool install --global dotnet-sonarscanner
- dotnet restore Project/Project.csproj
- dotnet restore Project.Tests/Project.Tests.csproj
before_script:
- export PATH="$PATH:$HOME/.dotnet/tools"
script:
- dotnet sonarscanner begin /k:"company:project" /n:"Project" /v:"#.#.#" /o:"companyname" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="$SONAR_KEY" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.cs.opencover.reportsPaths="lcov.opencover.xml" || true
- dotnet build Project/Project.csproj
- dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=../lcov
- dotnet sonarscanner end /d:sonar.login="$SONAR_KEY" || true
cache:
directories:
- '$HOME/.nuget/packages'
- '$HOME/.local/share/NuGet/Cache'
- '$HOME/.sonar/cache'
Change company , project , #.#.# and Project words with yours.
GitHub Actions
In order to keep secret the sonar key, create an environment variable on Settings -> Secrets called SONAR_KEY.
Create this file /.github/workflows/build.yml on your project:
name: build
on:
push:
branches: [master]
pull_request:
branches: [master]
env:
dotnet: 3.1.201
version: #.#.#
key: company:project
organization: company:project
name: Project
jobs:
build:
runs-on: ${{ matrix.platform }}
strategy:
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
name: build on ${{ matrix.platform }}
steps:
- uses: actions/checkout@v2
- name: setup .Net Core
uses: actions/setup-dotnet@v1
with:
dotnet-version: ${{ env.dotnet }}
- name: restore
run: dotnet restore Project/Project.csproj
- name: build
run: dotnet build Project/Project.csproj --no-restore
test:
runs-on: ubuntu-latest
name: test
steps:
- uses: actions/checkout@v2
- name: setup .Net Core
uses: actions/setup-dotnet@v1
with:
dotnet-version: ${{ env.dotnet }}
- name: install sonar-scanner
run: dotnet tool install --global dotnet-sonarscanner
- name: restore
run: dotnet restore Project/Project.csproj
- name: build
run: dotnet build Project/Project.csproj --no-restore
- name: restore test
run: dotnet restore Project.Tests/Project.Tests.csproj
- name: build test
run: dotnet build Project.Tests/Project.Tests.csproj --no-restore
- name: scanner begin
run: dotnet sonarscanner begin /k:"${{ env.key }}" /n:"${{ env.name }}" /v:"${{ env.version }}" /o:"${{ env.organization }}" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="${{ secrets.SONAR_TOKEN }}" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.cs.opencover.reportsPaths="lcov.opencover.xml"
- name: scanner build
run: dotnet build Project/Project.csproj
- name: scanner test
run: dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=../lcov
- name: scanner end
run: dotnet sonarscanner end /d:sonar.login="${{ secrets.SONAR_TOKEN }}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Change company , project , #.#.# and Project words with yours.
Commit and push the file into the master branch and see this script running on the Actions tab.
Test Case
If you want to see all this working together in a project, take a look at this GitHub repo Kata: TDD Arabic to Roman Numbers with C#.
That’s All Folks!
Happy Coding 🖖
Top comments (4)
IDK, because I've used the same configuration and debug tests on the three OS, and works fine.
Nice to know, thanks for sharing!
Nice Article. Is this possible to generate coverlet xml file using test project dll instead of csproj? Also, Is there any way to generate a single coverage file for multiple test case projects?
I've never tried scan a
dllfile and to be honest sound weird to me because this practice it's more to check quality on source code and adllfile, in general, have compiled code.On the coverage reports that I see on C# and JS, all of them create a report for each file and not for each test case.