Don't you want to save time?
I think it is common knowledge that testing applications is a crucial part of software development. It is also very helpful when it comes to developing separate features.
But what if your tests are very slow? Maybe a single test case takes 30 seconds to execute? Or your local test setup is so cumbersome you want to avoid it? You will end up not using them much, and this will harm your velocity.
Here are some traits of good tests in a project:
- You can easily run just one test
- Running just one test is very quick
- You don't need to do any changes to the environment to run tests locally
- You can run tests repeatedly without reinitializing the database
- You can run locally the same test suite that is run on your pipeline
- Even running the whole test suite is fast
In this post, I will focus on the points related to speed.
We really need speed
Just one time reiterating on this so it's clear - the speed of tests runtime will directly affect your delivery speed. And delivery is crucial.
But what we can do about our tests? Turns out there is one key change that can be applied to a lot of projects, is almost a drop-in replacement and usually results in an extreme speed boost.
How Jest does it
Most projects use Jest to run their tests. It's the default runtime for React apps and NestJS apps.
Assume we have the following NestJS application:
- Is Typescript based
- Integration tests start up the NestJS application module
- Supertest is used to hit the API
- Nest build takes 5 seconds
- Nest app startup takes 1 second
Let's see how Jest runs tests:
- Start a Typescript compiler instance and compile and run any global setup defined
- Find all files containing tests, most likely files ending with .spec.ts
- For every of found files:
- Start a Typescript compiler instance and compile the file, which in turns compiles the whole application because NestJS is imported to create the application to test
- Run the tests in the file
If we had 10 test files, it will result in at least 10 * 5, so 50 seconds wasted on recompiling the application over and over again 10 times, and then another 10 seconds to reinitialize the application 10 times.
The more files there are, the slower it gets.
Why does Jest do that? Probably due to sandboxing, maybe it makes sense. Parallelization works fine and improves on that but be prepared to bump your runners 1 or 2 tiers up to make them handle parallelized compilation over and over again.
The secret sauce - Mocha
Mocha (https://mochajs.org/) is a test runtime just like Jest. But it does things a bit differently.
Assuming the same setup we had in Jest example, here is what Mocha would do:
- Find all files containing tests, most likely files ending with .spec.ts
- Start a Typescript compiler instance and compile all tests at once like all of them were included in the tsconfig project
- Run your global setup
- Find all
describe
blocks in compiled tests - For every
describe
:- Run the describe block, of course effectively running the tests
The key here is that compilation is happening only once. If you move your application setup to global setup you can also initialize the application just one time and reuse it.
This will bring the same test runtime down to 5 + 1 seconds, making it over 8x faster. And adding more files won't affect the performance much.
How to migrate if I already use Jest?
This boils down to several steps:
- Install Mocha, Chai and Sinon - Chai is for assertions and Sinon is for mocking, you can skip Chai if you want to keep using Jest expect, which is possible
- Create mocha config file, you can use a template I made: https://pastebin.com/m2x3H2Mr
- Convert all of your
beforeAll
andafterAll
calls tobefore
andafter
- this is just a name change you can easily regex replace in all your tests - Convert all your Jest mocks to use Sinon, the usage is very similar
- If you want to use Jest's expect, you can import it in every test from Jest package or export it globally somewhere, otherwise rewrite your assertions to use one of Chai apis, I think
assert
one is the most similar to what Jest offers
2024 update
Chai is no longer recommended, best to use Jest Expect module instead!
The biggest issue would be to convert all your Jest mocks to sinon. Maybe there is a way to use Jest's mocks but rewriting all of them is pretty simple, and many cases can be regex replaced.
And that's it! After that, you are ready to run mocha
and enjoy your tests running much faster than before.
Happy testing!
Top comments (5)
Nice article.
I'm curious though. I understand the explanation of the algorithm with which each testing framework tests code as you have outlined. What I want to confirm is is you ran/have found any actual tests to prove that speed is higher in Mocha than in Jest.
Yes, I ported tests in several big projects at my previous work and at my current work, very recently, which actually gave me the idea to write the article. The gain in case of my recent rewrite was going down from 16 minutes to 3 minutes on GitLab CI, and on development machines, running just tests, from 200 seconds to 51 seconds. The more granularly your tests are separated into separate files the higher the gain will be, and the longer your initia setup takes the higher the gain will be as well. I see that also many people recommend using swc, but last time I checked it had troubles with decorators that are used extensively in NestJS, and I really really want to run tests with the same compiler that will finally compile my app for production usage. Hope it helps! I suggest you give it a try, mocha is awesome.
Incredible. Thank you
Nice article!
Have the similar issues and thinking about switching to mocha.
Considering your article, several others and this issue I'm just wondering why everyone claims that Jest is faster? It's slow even on my relatively small amount of tests.
I don't know, but I think this might come from people running tests written in pure JS on pure JS apps, so the TypeScript compiler doesn't need to do anything. This would be pretty common on the frontend before TS really got widely used.