PITest - How mutation can help for Unit tests.

PITest is a Java library that executes mutations on your JUnit tests to check if the coverage is correct.

What is mutation testing?

Mutation testing is a test process to check if your tests are checking the right things by modifying some elements in the code. If at least a test fails with a mutation, the mutation is killed. Otherwise, it stays alive.

The objective is to have all mutation tests killed.

For example, if you have a condition in your method, one mutator will change the condition by true. So if you don't check which methods are called or the value which must be returned, the mutation will live.

How to configure it?

You just have to import the library in your pom and declare it in the build section.

<project>
    ...
    <build>
        <plugins>
            <plugin>
                <groupId>org.pitest</groupId>
                <artifactId>pitest-maven</artifactId>
                <version>1.6.2</version>
                <executions>
                    <execution>
                        <id>pitest</id>
                        <phase>test</phase>
                        <goals>
                            <goal>mutationCoverage</goal>
                        </goals>
                    </execution>
                </executions>
                <dependencies>
                    <dependency>
                        <groupId>org.pitest</groupId>
                        <artifactId>pitest-junit5-plugin</artifactId>
                        <version>0.12</version>
                    </dependency>
                </dependencies>
                <configuration>
                    <timestampedReports>false</timestampedReports>
                    <mutationThreshold>100</mutationThreshold>
                    <coverageThreshold>80</coverageThreshold>
                </configuration>
            </plugin>
        </plugins>
    </build>
    ...
</project>

In this section, you will have all the configuration of the mutation testing like :

• mutationThreshold: Percentage of mutation that must work to success
• coverageThreshold: Percentage of the code coverage to succeed the build
• excludedClasses: Classes that have to be ignored by PITest.
• mutators: Mutation operator to execute on your project
• ...

Link to the PITest Configuration documentation

How to run it?

To run it alone you can use the next command

mvn org.pitest:pitest-maven:mutationCoverage

Otherwise, you can use

mvn verify

Some options can be provided to the command

mvn org.pitest:pitest-maven:scmMutationCoverage -DmutationThreshold=85

Detail of the other options available

• http://pitest.org/quickstart/commandline/
• http://pitest.org/quickstart/maven/

Results

If the mutation threshold is not obtained, your build will fail and it will quickly say what is the problem.

To see properly where is the issue, at every build PITest generates an HTML report to target/pit-reports/YYYYMMDDHHMI! With this element, you can quickly check the code and mutation coverage for each class.

And if you go to the class detail, you will see the errors and their details.

Documentation links :

• Official Website
• Tutorial with PITest
• GitHub Project

I hope it will help you!