In this guide, I'm telling you:
- How you can link JUnit 5 tests with issues in your task tracker systems?
- How to generate documentation based on it automatically?
- How to host the result documentation on GitHub Pages?
You can find the entire repository with code examples on GitHub by this link. The generated documentation also available here.
Issue Annotation
There is a cool library called JUnit Pioneer. It's an extension pack that includes some features that vanilla JUnit lacks. These are cartesian product tests, JSON argument parameterized source, retrying tests and many others. But I'm particularly interested in Issue annotation. Look at the code example below:
class TestExample {
@Test
@Issue("HHH-16417")
void testSum() {
...
}
@Test
@Issue("HHH-10000")
void testSub() {
...
}
}
I put actual task IDs from Hibernate task tracker to make result documentation more concise.
As you can see, we can add the @Issue
annotation with task ID that is associated with the test. So, every time you notice a test failure, you know what you have broken.
Setting up service loaders
JUnit Pioneer provides an API that allows to get information about tests that marked with @Issue
annotation. Meaning that we can combine the information in the HTML report and share it with other team members. For example, QA engineers might find it beneficial. Because now they are aware what tests do the project contains, and what bugs do they check.
There is a special interface IssueProcessor
. Its implementation acts like a callback. Look at the code snippet below:
public class SimpleIssueProcessor implements IssueProcessor {
@Override
public void processTestResults(List<IssueTestSuite> issueTestSuites) {
...
}
}
However, we also to need to set up SimpleIssueProcessor
as Java Service Loader. Otherwise, JUnit runner won’t register it. Create a new file with a name of org.junitpioneer.jupiter.IssueProcessor
in src/test/resources/META-INF/services
directory. It has to contain one row with the fully qualified name of the implementation (in our case, SimpleIssueProcessor
). Look at the code snippet below:
org.example.SimpleIssueProcessor
Besides, there is another service loader to register. It’s the one provided by JUnit Pioneer library that does the complex logic of parsing information and delegating control to IssueProcessor
implementation. Create a new file with a name of org.junit.platform.launcher.TestExecutionListener
in the same directory. Look at the required file content below:
org.junitpioneer.jupiter.issue.IssueExtensionExecutionListener
Now we’re ready. You can put println
statement in your IssueProcessor
implementation to check that the framework invokes it after tests’ execution.
Creating meta-information JSON file
The documentation generation process consists of two steps:
- Generate documentation in JSON format (because it's easy to parse).
- Put the information into HTML template.
Look at the SimpleIssueProcessor
code below:
public class SimpleIssueProcessor implements IssueProcessor {
@Override
@SneakyThrows
public void processTestResults(List<IssueTestSuite> issueTestSuites) {
writeFileToBuildFolder(
"test-issues-info.json",
new ObjectMapper().writeValueAsString(
issueTestSuites.stream()
.map(issueTestSuite -> Map.of(
"issueId", issueTestSuite.issueId(),
"tests", issueTestSuite.tests()
.stream()
.map(test -> parseTestId(test.testId()))
.toList()
))
.toList()
)
);
}
...
}
The
writeToBuildFolder
method creates a file by pathbuild/classes/java/test/test-issues-info.json
. I use Gradle, but if you prefer Maven, your path will differ a bit. You can check out the source code of the function by this link.
The result JSON is an array. Look at the generated example below:
[
{
"tests": [
{
"testId": "TestExample.testSum",
"urlPath": "org/example/TestExample.java#L12"
}
],
"issueId": "HHH-16417"
},
{
"tests": [
{
"testId": "TestExample.testSub",
"urlPath": "org/example/TestExample.java#L18"
}
],
"issueId": "HHH-10000"
}
]
There is an issue ID and set of tests that reference to it (theoretically, there might be several tests pointing the same issue).
Now we need to parse the required information from the supplied List<IssueTestSuite>
. Look at the parseTestId
function below.
@SneakyThrows
private static Map<String, Object> parseTestId(String testId) {
final var split = testId.split("/");
final var className = split[1].substring(7, split[1].length() - 1);
final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");
final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);
final var classPool = ClassPool.getDefault();
classPool.appendClassPath(new ClassClassPath(clazz));
final var methodLineNumber = classPool.get(className)
.getDeclaredMethod(method)
.getMethodInfo()
.getLineNumber(0);
return Map.of(
"testId", lastArrayElement(className.split("\\.")) + "." + method,
"urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
);
}
Let's deconstruct this code snippet step by step.
The library puts testId
as the string pattern below:
// [engine:junit-jupiter]/[class:org.example.TestExample]/[method:testSum()]
Firstly, we get fully qualified class name and method name. Look at the code below:
final var split = testId.split("/");
// [class:org.example.TestExample] => org.example.TestExample
final var className = split[1].substring(7, split[1].length() - 1);
// [method:testSum()] => testSum
final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");
Afterwards, we determine the line number of the test method. It’s useful to set links that point to the particular line of code. Look at the snippet below:
// Load test class
final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);
final var classPool = ClassPool.getDefault();
classPool.appendClassPath(new ClassClassPath(clazz));
final var methodLineNumber = classPool.get(className)
.getDeclaredMethod(method)
.getMethodInfo()
.getLineNumber(0);
ClassPool
comes from Javaassist library. It gives convenient API to retrieve the line number of Java method.
Here we perform these steps:
- Get the
Class
instance of the test suite. - Initialize
ClassPool
. - Append a test class to the pool
- Get the line number of the test method.
And finally, we put together the information chunks into java.util.Map
that we eventually convert to JSON. Look at the last piece of code below:
return Map.of(
// TestExample.testSum
"testId", lastArrayElement(className.split("\\.")) + "." + method,
// org/example/TestExample.java#L11
"urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
);
The testId
property is just a combination of a simple class name and test method name. Whilst urlPath
is part of the link on GitHub pointing to the specific line where we declared the test.
Generating documentation
Finally, it’s time to compose the generated JSON into a nicely laid out HTML page. Look at the entire snippet below. Then I’m explaining each part to you.
const fs = require('fs');
function renderIssues(issuesInfo) {
issuesInfo.sort((issueLeft, issueRight) => {
const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
return parseIssueId(issueRight) - parseIssueId(issueLeft);
})
return `
<table>
<tr>
<th>Issue</th>
<th>Test</th>
</tr>
${issuesInfo.flatMap(issue => issue.tests.map(test => `
<tr>
<td>
<a target="_blank" href="https://hibernate.atlassian.net/browse/${issue.issueId}">${issue.issueId}</a>
</td>
<td>
<a target="_blank" href="https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/${test.urlPath}">${test.testId}</a>
</td>
</tr>
`)).join('')}
</table>
`
}
console.log(`
<!DOCTYPE html>
<html lang="en">
<head>
<title>List of tests validation particular issues</title>
<meta charset="UTF-8">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
</head>
<body>
<h1>List of tests validation particular issues</h1>
<h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
${renderIssues(JSON.parse(fs.readFileSync('./build/classes/java/test/test-issues-info.json', 'utf8')))}
</body>
</html>
`)
I'm using Javascript and NodeJS runtime environment.
The renderIssues
function does the entire job. Let's deconstruct it step by step.
function renderIssues(issuesInfo) {
issuesInfo.sort((issueLeft, issueRight) => {
const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
return parseIssueId(issueRight) - parseIssueId(issueLeft);
})
...
}
The issuesInfo
is an array that we generated previously with IssueProcessor
. Therefore, each element has issueId
and tests belonging to it.
As long as each issue id has a format of MMM-123
we can sort them by number. In that case, we get issues sorted in descending order.
Look at the remaining portion of the function below:
const issueBaseUrl = "https://hibernate.atlassian.net/browse/";
const repoBaseUrl = "https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/"
return `
<table>
<tr>
<th>Issue</th>
<th>Test</th>
</tr>
${issuesInfo.flatMap(issue => issue.tests.map(test => `
<tr>
<td>
<a target="_blank" href="${issueBaseUrl}${issue.issueId}">${issue.issueId}</a>
</td>
<td>
<a target="_blank" href="${repoBaseUrl}${test.urlPath}">${test.testId}</a>
</td>
</tr>
`)).join('')}
</table>
`
Each present combination of issue and test transforms into a table row. Also, those snippets aren’t just plain text but links. You can open issue description and test declaration by clicking on it. Cool, isn’t it?
Then comes the output. Look at the final part of the script below:
console.log(`
<!DOCTYPE html>
<html lang="en">
<head>
<title>List of tests validation particular issues</title>
<meta charset="UTF-8">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
</head>
<body>
<h1>List of tests validation particular issues</h1>
<h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
${renderIssues(JSON.parse(
fs.readFileSync('./build/classes/java/test/test-issues-info.json',
'utf8')))}
</body>
</html>
`)
I write the output to console because later I redirect it to file.
The style sheet is called Tacit CSS. This is a set of CSS rules applied automatically. If you need to format an HTML page but don’t want to deal with complex layout, that’s a perfect solution.
The idea is to put the generated HTML table into a predefined template.
Setting up GitHub Pages
The documentation is no use until you can examine it. So, let's host it on GitHub Pages. Look at the pipeline below:
name: Java CI with Gradle
on:
push:
branches: [ "master" ]
permissions:
contents: read
pages: write
id-token: write
jobs:
build-and-deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
- name: Build with Gradle
run: ./gradlew build
- name: Set up NodeJS
uses: actions/setup-node@v3
with:
node-version: 16
- name: Run docs generator
run: ./docsGeneratorScript.sh
- name: Upload artifact
uses: actions/upload-pages-artifact@v1
with:
path: public/
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v1
The steps are:
- Set up JDK 17
- Build the project
- Set up NodeJS
- Generate documentation with the previously shown JS program
- Deploy the result to GitHub Pages
The docsGeneratorScript.sh
file is a trivial bash script. Look at its definition below:
mkdir -p public
touch ./public/index.html
node ./generateDocs.js > ./public/index.html
And that’s it! Now the documentation is available and being updated automatically each time somebody merges a pull request.
Conclusion
That’s all I wanted to tell you about linking tests with issues and generating documentation for it. If you have questions or suggestions, leave your comments down below. Thanks for reading!
Top comments (3)
Hello there,
This is a really nice post, but this is missing to handle parameterized tests, I'm not sure if there is any other, in case of parameterized test the testId's pattern is the following:
It will include the invocation of the test, so you can determine which of the tests from the multiple is running, could be valuable, but for the report you might want to just put one method into the list not all, so in this case I replaced the Map with a new type, where the equals is overwritten so I can use the distinct of the stream.
It looks a bit weird, but for a quick try it will do:
Btw, I was converting the JSON file to an AsciiDoc because I'm more into writing these, and I used JBang, as it has a maven plugin it can easily wired into the maven build and no other stack dependency is needed like NodeJS, but inside that it is at least easier to put together :D
Thank you for the post, this is a good one!
Thank you very much for such a detailed response. I'll take into account the information about parameterized tests
Thx for the post, a really complete idea. It comes in mind [ concordion.org/ ] maybe a contribution can be a good thing!
Thx for your time!
🖖peace🖖