DEV Community

loading...

Why I migrated from Kotest to JUnit

neelkamath profile image Neel Kamath ・2 min read

I used to use JUnit via the kotlin.test package a year ago, and then moved to JUnit via Kotest. However, due to an increasing number of issues with Kotest, many of which took me several hours to diagnose, I've gone back to JUnit via kotlin.test. It took me 10 hours to migrate several hundred tests, but almost all of it was mindless copy-pasting. Hopefully, you can make a more informed decision using this comparison:

  • Though Kotest has more matchers, they aren't type safe, and rarely pretty print. The lack of pretty printing means that you almost always have to debug the test case regardless of whether a more expressive matcher was used, thereby removing most of the matcher's use.
  • Gradle's --tests filter only works for top-level classes. This causes tests to take magnitudes longer to execute since the entire suite has to be run every time. Kotest does have filtering options through syntax and a plugin, but neither are feasible. The syntax requires you to find the test case, edit it, recompile the code, and remember to remove it later. Also, the syntax doesn't work for most test cases. As for the plugin, it's currently extremely buggy, and won't be of much help even after it improves because it can't be used to execute tests in a nontrivial set up. Similar to the way IntelliJ can't use gutter icons for JUnit tests if the app is running in a nontrivial Docker Compose setup, the Kotest plugin doesn't work either.
  • It's impossible to edit tests in the IDE because it takes twenty seconds to buffer one keystroke since Kotest requires nested lambdas spanning hundreds of lines of code. This syntax style offers no benefit to the programmer, and is also impossible to use in an IDE.
  • It incorrectly reports the number of passed test cases because it counts containers as test cases (e.g., it'll report running 500 tests even though there were only 300).
  • It incorrectly reports failed test cases because it reports tests in a manner Gradle doesn't understand (e.g., it'll report five failures even though there were only two).
  • The kotlin.test package requires pretty much the same amount of boilerplate Kotest does, but is easier to learn, easier to use, makes it easier to migrate to other frameworks, and has better multiplatform support. Besides, you'll have to learn kotlin.test's and JUnit's basics regardless of whether you're using Kotest since kotlin.test is bundled with Kotlin tests, and Kotest transparently uses JUnit.
  • Though JUnit doesn't have an easy way to run tear down code after all the test suites have executed, Kotest's ProjectListener.afterProject doesn't get run when using Gradle's --fail-fast flag.

Discussion (3)

pic
Editor guide
Collapse
sksamuel profile image
Stephen Samuel • Edited

It incorrectly reports the number of passed test cases because it counts containers as test cases <--- gradle counts them as test cases, not kotest. It's just running on the JUnit test platform.

It incorrectly reports failed test cases because it reports tests in a manner Gradle doesn't understand <--- same thing.

Once gradle has proper support for nested clases it'll go away. If that ever happens. Hence why this has been started: github.com/kotest/kotest-gradle-pl...

Collapse
neelkamath profile image
Neel Kamath Author

Sorry if it came across as Kotest's particular issue. The reason I wrote it as if Kotest and Gradle were one thing was because Kotest currently uses Gradle (and hence has the same shortcomings, but JUnit doesn't).

Collapse
sksamuel profile image
Stephen Samuel

Its all good, feedback is always useful.