First Mocked Unit Test Generated by DDTJ - Building DDTJ Day 9

Yesterday I ran into some snags, but I'm glad to report that this is all behind me. We finally have the first DDT generated test in history!

I’d like to proclaim victory, but the road ahead is still long and the result is “underwhelming” at this point. We have generated code, but it still doesn’t compile right away and at least for POJOs it doesn’t inject the mocks (although it creates them). I think those are surmountable problems that we can solve moving forward.

But I’m running a bit ahead. Let’s talk about what’s happening with the code right now… Or at least in the current PR that’s still waiting for more test coverage.

Right now I’m running a super trivial application:

public class BasicApp {
   private BasicDependency dependency = new BasicDependency();
   public static void main(String[] args) throws InterruptedException {
       new BasicApp().run();
   }

   public void run() throws InterruptedException {
       System.out.println("This is the first testable method");
       System.out.println(dependency.otherMethod("This prints three", 3));
   }
}

Which invokes:

public class BasicDependency {
   public String otherMethod(String key, int counter) throws InterruptedException {
       System.out.println("otherMethod");
       return key + " " + counter;
   }
}

To generate a test for this, I used the following process:

Run the Backend Server

The server process collects the execution data from the app:

java -jar target/Backend-0.0.5-SNAPSHOT.jar

Run the App

The next stage is getting the server to run/debug the application. This replaces the standard java command line with something like this:

java -jar target/CLI-0.0.5-SNAPSHOT-shaded.jar -run dev.ddtj.backend.testdata.BasicApp -classpath Backend/target/test-classes

Notice this command is asynchronous since it runs in the backend server context.

Find and Generate the Test

First, we need to list the classes on which we have instrumentation:

java -jar target/CLI-0.0.5-SNAPSHOT-shaded.jar -c

Which prints out:

Class Name                                                  | Method Count    | Execution Count
----------                                                  | ------------    | ---------------
dev.ddtj.backend.testdata.BasicDependency                   | 1               | 1
dev.ddtj.backend.testdata.BasicApp                          | 2               | 2

Next, we need to see the methods we can test in a specific class:

java -jar target/CLI-0.0.5-SNAPSHOT-shaded.jar -m dev.ddtj.backend.testdata.BasicApp

Which prints out:

Method Name                                                 | Total Execution
-----------                                                 | ---------------
main([Ljava/lang/String;)V                                  | 1
run()V                                                      | 1

Now we need to find the tests available. When we have one test, it seems redundant, but it makes sense when we have more:

java -jar target/CLI-0.0.5-SNAPSHOT-shaded.jar -t "dev.ddtj.backend.testdata.BasicApp.run()V"

Prints out:

Test ID                                 | Test Hour of the Day
-------                                 | --------------------
JuZlJX5ERAKXZQR8CpYgiA--7               | Thu Dec 30 08:31:25 IST 2021

We can now generate a test based on these results:

java -jar target/CLI-0.0.5-SNAPSHOT-shaded.jar -g "dev.ddtj.backend.testdata.BasicApp,run()V,JuZlJX5ERAKXZQR8CpYgiA--7"

Which prints out:

/**
 * Generated by <a href="https://github.com/ddtj/ddtj/">ddtj</a>
 */
package dev.ddtj.backend.testdata.test;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import dev.ddtj.backend.testdata.BasicApp;
import dev.ddtj.backend.testdata.BasicDependency;

@ExtendWith(MockitoExtension.class)
class BasicAppTests {
    @Test
    void runTest() {
        BasicDependency BasicDependencyMock = Mockito.mock(BasicDependency.class);
        Mockito.lenient().when(BasicDependencyMock.otherMethod("This prints three", 3)).thenReturn("This prints three 3");
        BasicApp myObjectInstance =  new BasicApp();
        myObjectInstance.run();
    }
}

What Isn’t Working?

There are several problems in the code above. First, we don’t detect checked exceptions, so this code won’t compile.

As a short-term workaround, I’m adding a “throws Exception” to the test method. This ensures it compiles properly and fails as expected.

The bigger problem is that the mock isn’t properly bound. I expected this since I didn’t write the code for that and also there’s technically no way to bind that mock. I hope this will work more reasonably when testing on “real world” code.

The code doesn’t use the @Mock or @InjectMocks annotations. I think that’s something that I can improve on. I’d like to have multiple styles of test generation to support various personal tastes.

Since the mock isn’t invoked, the test would fail on that, but I added the `lenient() call as a short-term workaround so we can move forward.

What’s Working?

I’m so pleased it generated the mock code correctly and implemented the mock.

Object creation and dependencies included a lot of hairy code, but it’s coming together now and I think the basis is very good.

I’m very pleased with the data collection code and the basic architecture. Now the key challenge is fine tuning and scaling this so it works correctly for more “real world” workloads.

I think I’ll go into greater details on this in my postmortem blog post next week. I think I need to give this some time to see how everything fits together.

Tomorrow

Right now I have two big challenges ahead of me:

1. Merge the PR - this will be hard, especially testing all this code I wrote
2. Scale and improve to use cases

At this stage, I’m pretty happy that I accomplished the basic goal. But I want to build a proper MVP so I hope I can get a real world application running. I think blue sky initiatives like a web interface, would have to wait until after this stage.

If you want to keep up with the latest updates on this series and the many other things I work on, then follow me on twitter.