How to Measure Code Coverage in a Running Java Application Using JaCoCo

When developing software, it’s crucial to know how much of your codebase is covered by tests. This article will walk you through using the JaCoCo (Java Code Coverage) tool to measure code coverage in a running Java application. We’ll explore how JaCoCo helps find gaps in coverage and ensures you're not missing critical parts of your application that need testing.

What is Code Coverage?

Code coverage is a metric that determines what percentage of your code is being executed by tests. While it doesn't tell you if your tests are well-written, it shows which parts of your code haven't been tested. Low coverage means there are areas of the code that haven’t been executed, and these areas may require more attention. However, even with 100% coverage, your tests might still miss important logic or edge cases. Code coverage simply shows that certain lines of code have been executed at least once during testing.

Code coverage helps find blind spots in testing, especially in large codebases or when tests are written after the application is already developed.

Why Use JaCoCo for Code Coverage?

JaCoCo is one of the most popular code coverage tools for Java applications. It works by instrumenting the bytecode of your application at runtime, allowing you to gather coverage data while your application is running, even in production environments. JaCoCo supports both unit tests and functional tests.

Using JaCoCo in a live environment measures which parts of your code are executed as users interact with your application. This is particularly useful for seeing how well your functional or integration tests cover the actual business logic during runtime.

Why This Approach Is Not a Replacement for Traditional Test Management

It’s important to note that code coverage is not a replacement for a traceability matrix or other test management techniques. A traceability matrix ensures that all requirements and scenarios are covered, while code coverage only identifies the parts of your code that haven’t been executed by tests.

Although code coverage helps identify gaps, having 100% coverage doesn’t guarantee that all scenarios are tested or that your application is free of defects. Zero coverage, however, indicates that specific parts of the code were not tested and need immediate attention.

Using JaCoCo to Measure Code Coverage at Runtime

Note: This approach is particularly useful for measuring code coverage during functional or integration testing, rather than unit testing, as it tracks code executed during real interactions with the application.

Step 1: Download JaCoCo Tools

Before you start, you'll need to download JaCoCo tools.

To download the required JaCoCo tools, run the following curl commands:

curl -O https://repo1.maven.org/maven2/org/jacoco/org.jacoco.cli/0.8.12/org.jacoco.cli-0.8.12-nodeps.jar

curl -O https://repo1.maven.org/maven2/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar

Step 2: Start Your Java Application with JaCoCo

To instrument your Java application with JaCoCo, you need to start the application with the JaCoCo agent. This agent monitors and records the coverage while the application is running. Here's how to start your application with JaCoCo:

java '-javaagent:org.jacoco.agent-0.8.12-runtime.jar=port=6300,address=0.0.0.0,destfile=jacoco.exec,includes=com.example.*,append=true,output=tcpserver' -jar build/libs/demo_api_server-0.0.1-SNAPSHOT.jar

Breaking Down the Command:

• -javaagent: Tells the JVM to run JaCoCo as an agent alongside your application.
• org.jacoco.agent-0.8.12-runtime.jar: The path to the JaCoCo agent.
• port=6300: JaCoCo listens on port 6300 for requests to dump coverage data.
• address=0.0.0.0: Sets the address that the JaCoCo agent will bind to.
• destfile=jacoco.exec: Specifies the file where coverage data will be stored.
• includes=com.example.*: Limits coverage to specific packages (in this case, com.example.*).
• append=true: Appends new coverage data to the existing file.
• output=tcpserver: Specifies that JaCoCo will be available via a TCP server, allowing you to dump coverage data on demand.

Step 3: Dump the Coverage Data

Once your application is running and you’ve executed the desired test scenarios (manually or automatically), you can trigger JaCoCo to dump the coverage data. This is particularly useful in production or staging environments where you don't want to shut down the application.

Use the following command to dump the coverage data:

java -jar org.jacoco.cli-0.8.12-nodeps.jar dump --address localhost --port 6300 --destfile jacoco.exec --reset

• dump: Extracts the coverage data from the running JVM.
• --address localhost: The address where JaCoCo is listening.
• --port 6300: The port where JaCoCo is serving coverage data.
• --destfile jacoco.exec: The file where the dumped data will be saved.
• --reset: Clears the coverage data after dumping it.

Step 4: Generate the Coverage Report

After dumping the coverage data, generate the coverage report using JaCoCo’s CLI tool. Here’s how to create an HTML report:

java -jar org.jacoco.cli-0.8.12-nodeps.jar report jacoco.exec --classfiles build/classes/java/main --sourcefiles src/main/java --html coverage_report

This command will generate an HTML report showing which parts of the code were executed.

Step 5: Analyze the Results

Once you have the coverage report, examine areas with zero coverage and identify testing gaps. Keep in mind that high coverage doesn’t guarantee good tests, but no coverage indicates missing tests that need immediate attention.

You can integrate JaCoCo reports into your CI pipeline (e.g., Jenkins, GitLab CI, GitHub Actions) to track code coverage over time.

Key Takeaways

• Code coverage measures how much of your code is executed by tests, but it doesn’t guarantee test quality.
• JaCoCo is an easy-to-use tool to measure runtime code coverage, especially useful for identifying gaps in functional or manual testing.
• Code coverage and traceability matrix serve different purposes: the former highlights executed code, while the latter ensures test scenarios align with requirements.
• For large projects, tools like JaCoCo help quickly identify untested areas, but they should be combined with other testing practices.

By following the steps outlined in this article, you can effectively measure and analyze code coverage in your Java application, ensuring that critical parts of your code are tested.

Demo code

You can find the complete demo code for measuring code coverage using JaCoCo here:
https://github.com/arturmkr/jacoco-code-coverage-demo