I used to use JUnit via the
kotlin.test package a year ago, and then moved to JUnit via Kotest. However, due to an increasing number of issues with Kotest, many of which took me several hours to diagnose, I've gone back to JUnit via
kotlin.test. It took me 10 hours to migrate several hundred tests, but almost all of it was mindless copy-pasting. Hopefully, you can make a more informed decision using this comparison:
- Though Kotest has more matchers, they aren't type safe, and rarely pretty print. The lack of pretty printing means that you almost always have to debug the test case regardless of whether a more expressive matcher was used, thereby removing most of the matcher's use.
--testsfilter only works for top-level classes. This causes tests to take magnitudes longer to execute since the entire suite has to be run every time. Kotest does have filtering options through syntax and a plugin, but neither are feasible. The syntax requires you to find the test case, edit it, recompile the code, and remember to remove it later. Also, the syntax doesn't work for most test cases. As for the plugin, it's currently extremely buggy, and won't be of much help even after it improves because it can't be used to execute tests in a nontrivial set up. Similar to the way IntelliJ can't use gutter icons for JUnit tests if the app is running in a nontrivial Docker Compose setup, the Kotest plugin doesn't work either.
- It's impossible to edit tests in the IDE because it takes twenty seconds to buffer one keystroke since Kotest requires nested lambdas spanning hundreds of lines of code. This syntax style offers no benefit to the programmer, and is also impossible to use in an IDE.
- It incorrectly reports the number of passed test cases because it counts containers as test cases (e.g., it'll report running 500 tests even though there were only 300).
- It incorrectly reports failed test cases because it reports tests in a manner Gradle doesn't understand (e.g., it'll report five failures even though there were only two).
kotlin.testpackage requires pretty much the same amount of boilerplate Kotest does, but is easier to learn, easier to use, makes it easier to migrate to other frameworks, and has better multiplatform support. Besides, you'll have to learn
kotlin.test's and JUnit's basics regardless of whether you're using Kotest since
kotlin.testis bundled with Kotlin tests, and Kotest transparently uses JUnit.
- Though JUnit doesn't have an easy way to run tear down code after all the test suites have executed, Kotest's
ProjectListener.afterProjectdoesn't get run when using Gradle's