DEV Community

Semyon Kirekov
Semyon Kirekov

Posted on

JUnit 5: link tests with task tracker issues

#java #documentation #testing #githubactions

In this guide, I'm telling you:

  1. How you can link JUnit 5 tests with issues in your task tracker systems?
  2. How to generate documentation based on it automatically?
  3. How to host the result documentation on GitHub Pages?

You can find the entire repository with code examples on GitHub by this link. The generated documentation also available here.

Article meme cover

Issue Annotation

There is a cool library called JUnit Pioneer. It's an extension pack that includes some features that vanilla JUnit lacks. These are cartesian product tests, JSON argument parameterized source, retrying tests and many others. But I'm particularly interested in Issue annotation. Look at the code example below:

class TestExample {
    @Test
    @Issue("HHH-16417")
    void testSum() {
        ...
    }

    @Test
    @Issue("HHH-10000")
    void testSub() {
        ...
    }
}
Enter fullscreen mode Exit fullscreen mode

I put actual task IDs from Hibernate task tracker to make result documentation more concise.

As you can see, we can add the @Issue annotation with task ID that is associated with the test. So, every time you notice a test failure, you know what you have broken.

Setting up service loaders

JUnit Pioneer provides an API that allows to get information about tests that marked with @Issue annotation. Meaning that we can combine the information in the HTML report and share it with other team members. For example, QA engineers might find it beneficial. Because now they are aware what tests do the project contains, and what bugs do they check.

There is a special interface IssueProcessor. Its implementation acts like a callback. Look at the code snippet below:

public class SimpleIssueProcessor implements IssueProcessor {
    @Override
    public void processTestResults(List<IssueTestSuite> issueTestSuites) {
        ...
    }
}
Enter fullscreen mode Exit fullscreen mode

However, we also to need to set up SimpleIssueProcessor as Java Service Loader. Otherwise, JUnit runner won’t register it. Create a new file with a name of org.junitpioneer.jupiter.IssueProcessor in src/test/resources/META-INF/services directory. It has to contain one row with the fully qualified name of the implementation (in our case, SimpleIssueProcessor). Look at the code snippet below:

org.example.SimpleIssueProcessor
Enter fullscreen mode Exit fullscreen mode

Besides, there is another service loader to register. It’s the one provided by JUnit Pioneer library that does the complex logic of parsing information and delegating control to IssueProcessor implementation. Create a new file with a name of org.junit.platform.launcher.TestExecutionListener in the same directory. Look at the required file content below:

org.junitpioneer.jupiter.issue.IssueExtensionExecutionListener
Enter fullscreen mode Exit fullscreen mode

Now we’re ready. You can put println statement in your IssueProcessor implementation to check that the framework invokes it after tests’ execution.

Creating meta-information JSON file

The documentation generation process consists of two steps:

  1. Generate documentation in JSON format (because it's easy to parse).
  2. Put the information into HTML template.

Look at the SimpleIssueProcessor code below:

public class SimpleIssueProcessor implements IssueProcessor {
    @Override
    @SneakyThrows
    public void processTestResults(List<IssueTestSuite> issueTestSuites) {
        writeFileToBuildFolder(
            "test-issues-info.json",
            new ObjectMapper().writeValueAsString(
                issueTestSuites.stream()
                    .map(issueTestSuite -> Map.of(
                        "issueId", issueTestSuite.issueId(),
                        "tests", issueTestSuite.tests()
                                     .stream()
                                     .map(test -> parseTestId(test.testId()))
                                     .toList()
                    ))
                    .toList()
            )
        );
    }
    ...
}
Enter fullscreen mode Exit fullscreen mode

The writeToBuildFolder method creates a file by path build/classes/java/test/test-issues-info.json. I use Gradle, but if you prefer Maven, your path will differ a bit. You can check out the source code of the function by this link.

The result JSON is an array. Look at the generated example below:

[
  {
    "tests": [
      {
        "testId": "TestExample.testSum",
        "urlPath": "org/example/TestExample.java#L12"
      }
    ],
    "issueId": "HHH-16417"
  },
  {
    "tests": [
      {
        "testId": "TestExample.testSub",
        "urlPath": "org/example/TestExample.java#L18"
      }
    ],
    "issueId": "HHH-10000"
  }
]
Enter fullscreen mode Exit fullscreen mode

There is an issue ID and set of tests that reference to it (theoretically, there might be several tests pointing the same issue).

Now we need to parse the required information from the supplied List<IssueTestSuite>. Look at the parseTestId function below.

@SneakyThrows
private static Map<String, Object> parseTestId(String testId) {
    final var split = testId.split("/");
    final var className = split[1].substring(7, split[1].length() - 1);
    final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");
    final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);

    final var classPool = ClassPool.getDefault();
    classPool.appendClassPath(new ClassClassPath(clazz));
    final var methodLineNumber = classPool.get(className)
                                     .getDeclaredMethod(method)
                                     .getMethodInfo()
                                     .getLineNumber(0);
    return Map.of(
        "testId", lastArrayElement(className.split("\\.")) + "." + method,
        "urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
    );
}
Enter fullscreen mode Exit fullscreen mode

Let's deconstruct this code snippet step by step.

The library puts testId as the string pattern below:

// [engine:junit-jupiter]/[class:org.example.TestExample]/[method:testSum()]
Enter fullscreen mode Exit fullscreen mode

Firstly, we get fully qualified class name and method name. Look at the code below:

final var split = testId.split("/");
// [class:org.example.TestExample] => org.example.TestExample
final var className = split[1].substring(7, split[1].length() - 1);
// [method:testSum()] => testSum
final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");
Enter fullscreen mode Exit fullscreen mode

Afterwards, we determine the line number of the test method. It’s useful to set links that point to the particular line of code. Look at the snippet below:

// Load test class
final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);

final var classPool = ClassPool.getDefault();
classPool.appendClassPath(new ClassClassPath(clazz));

final var methodLineNumber = classPool.get(className)
                                 .getDeclaredMethod(method)
                                 .getMethodInfo()
                                 .getLineNumber(0);
Enter fullscreen mode Exit fullscreen mode

ClassPool comes from Javaassist library. It gives convenient API to retrieve the line number of Java method.

Here we perform these steps:

  1. Get the Class instance of the test suite.
  2. Initialize ClassPool.
  3. Append a test class to the pool
  4. Get the line number of the test method.

And finally, we put together the information chunks into java.util.Map that we eventually convert to JSON. Look at the last piece of code below:

return Map.of(
    // TestExample.testSum
    "testId", lastArrayElement(className.split("\\.")) + "." + method,
    // org/example/TestExample.java#L11
    "urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
);
Enter fullscreen mode Exit fullscreen mode

The testId property is just a combination of a simple class name and test method name. Whilst urlPath is part of the link on GitHub pointing to the specific line where we declared the test.

Generating documentation

Finally, it’s time to compose the generated JSON into a nicely laid out HTML page. Look at the entire snippet below. Then I’m explaining each part to you.

const fs = require('fs');

function renderIssues(issuesInfo) {
  issuesInfo.sort((issueLeft, issueRight) => {
    const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
    return parseIssueId(issueRight) - parseIssueId(issueLeft);
  })
  return `
            <table>
                <tr>
                    <th>Issue</th>
                    <th>Test</th>
                </tr>
                ${issuesInfo.flatMap(issue => issue.tests.map(test => `
                    <tr>
                        <td>
                            <a target="_blank" href="https://hibernate.atlassian.net/browse/${issue.issueId}">${issue.issueId}</a>
                        </td>
                        <td>
                            <a target="_blank" href="https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/${test.urlPath}">${test.testId}</a>
                        </td>
                    </tr>
                `)).join('')}
            </table>
        `
}

console.log(`
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <title>List of tests validation particular issues</title>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
    </head>
    <body>
        <h1>List of tests validation particular issues</h1>
        <h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
        ${renderIssues(JSON.parse(fs.readFileSync('./build/classes/java/test/test-issues-info.json', 'utf8')))}
    </body>
    </html>
`)
Enter fullscreen mode Exit fullscreen mode

I'm using Javascript and NodeJS runtime environment.

The renderIssues function does the entire job. Let's deconstruct it step by step.

function renderIssues(issuesInfo) {
  issuesInfo.sort((issueLeft, issueRight) => {
    const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
    return parseIssueId(issueRight) - parseIssueId(issueLeft);
  })
  ...
}
Enter fullscreen mode Exit fullscreen mode

The issuesInfo is an array that we generated previously with IssueProcessor. Therefore, each element has issueId and tests belonging to it.

As long as each issue id has a format of MMM-123 we can sort them by number. In that case, we get issues sorted in descending order.

Look at the remaining portion of the function below:

const issueBaseUrl = "https://hibernate.atlassian.net/browse/";
const repoBaseUrl = "https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/"
  return `
            <table>
                <tr>
                    <th>Issue</th>
                    <th>Test</th>
                </tr>
                ${issuesInfo.flatMap(issue => issue.tests.map(test => `
                    <tr>
                        <td>
                            <a target="_blank" href="${issueBaseUrl}${issue.issueId}">${issue.issueId}</a>
                        </td>
                        <td>
                            <a target="_blank" href="${repoBaseUrl}${test.urlPath}">${test.testId}</a>
                        </td>
                    </tr>
                `)).join('')}
            </table>
        `
Enter fullscreen mode Exit fullscreen mode

Each present combination of issue and test transforms into a table row. Also, those snippets aren’t just plain text but links. You can open issue description and test declaration by clicking on it. Cool, isn’t it?

Then comes the output. Look at the final part of the script below:

console.log(`
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <title>List of tests validation particular issues</title>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
    </head>
    <body>
        <h1>List of tests validation particular issues</h1>
        <h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
        ${renderIssues(JSON.parse(
    fs.readFileSync('./build/classes/java/test/test-issues-info.json',
        'utf8')))}
    </body>
    </html>
`)
Enter fullscreen mode Exit fullscreen mode

I write the output to console because later I redirect it to file.

The style sheet is called Tacit CSS. This is a set of CSS rules applied automatically. If you need to format an HTML page but don’t want to deal with complex layout, that’s a perfect solution.

The idea is to put the generated HTML table into a predefined template.

Setting up GitHub Pages

The documentation is no use until you can examine it. So, let's host it on GitHub Pages. Look at the pipeline below:

name: Java CI with Gradle

on:
  push:
    branches: [ "master" ]

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-and-deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 17
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: 'temurin'
      - name: Build with Gradle
        run: ./gradlew build
      - name: Set up NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 16
      - name: Run docs generator
        run: ./docsGeneratorScript.sh
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v1
        with:
          path: public/
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v1
Enter fullscreen mode Exit fullscreen mode

The steps are:

  1. Set up JDK 17
  2. Build the project
  3. Set up NodeJS
  4. Generate documentation with the previously shown JS program
  5. Deploy the result to GitHub Pages

The docsGeneratorScript.sh file is a trivial bash script. Look at its definition below:

mkdir -p public
touch ./public/index.html
node ./generateDocs.js > ./public/index.html
Enter fullscreen mode Exit fullscreen mode

And that’s it! Now the documentation is available and being updated automatically each time somebody merges a pull request.

Conclusion

That’s all I wanted to tell you about linking tests with issues and generating documentation for it. If you have questions or suggestions, leave your comments down below. Thanks for reading!

Resources

  1. JUnit 5
  2. GitHub Pages
  3. The repository with source code
  4. The result generated documentation
  5. JUnit Pioneer
  6. Issue annotation from JUnit Pioneer
  7. Hibernate task tracker
  8. Java Service Loader
  9. Javaassist library
  10. Tacit CSS

Top comments (3)

Collapse
 
nandorholozsnyak profile image
Nándor Holozsnyák •

Hello there,

This is a really nice post, but this is missing to handle parameterized tests, I'm not sure if there is any other, in case of parameterized test the testId's pattern is the following:

[engine:junit-jupiter]/[class:org.example.SumTest]/[test-template:getSum(java.lang.Integer)]/[test-template-invocation:#1]
Enter fullscreen mode Exit fullscreen mode

It will include the invocation of the test, so you can determine which of the tests from the multiple is running, could be valuable, but for the report you might want to just put one method into the list not all, so in this case I replaced the Map with a new type, where the equals is overwritten so I can use the distinct of the stream.

It looks a bit weird, but for a quick try it will do:

final String method;
if (split[2].startsWith("[method:")) {
    // [method:testSum()] => testSum
    method = split[2].substring(8, split[2].length() - 1)
                     .replaceAll("\\(.*\\)", "");
} else {
    // [test-template:testSum(java.lang.Integer)] => testSum
    method = split[2].substring(15, split[2].length() - 1)
                     .replaceAll("\\(.*\\)", "");
}
Enter fullscreen mode Exit fullscreen mode

Btw, I was converting the JSON file to an AsciiDoc because I'm more into writing these, and I used JBang, as it has a maven plugin it can easily wired into the maven build and no other stack dependency is needed like NodeJS, but inside that it is at least easier to put together :D

Thank you for the post, this is a good one!

Collapse
 
kirekov profile image
Semyon Kirekov •

Thank you very much for such a detailed response. I'll take into account the information about parameterized tests

Collapse
 
stefanofago73 profile image
Stefano Fago •

Thx for the post, a really complete idea. It comes in mind [ concordion.org/ ] maybe a contribution can be a good thing!
Thx for your time!
🖖peace🖖

Read next

specialbroccoli profile image

AI Test Report - Summarize Failed Tests

SpecialBroccoli -

mickeystreicher profile image

Vedro Hooks

Mickey Streicher -

vkazulkin profile image

Spring Boot 3 application on AWS Lambda - Part 13 Measuring cold and warm starts with GraalVM Native Image

Vadym Kazulkin -

mspilari profile image

Building a Simple Voucher System for Small Businesses

Matheus Bernardes Spilari -