Originally posted on ivantanev.com
TL;DR
When you have Jest as your test runner, passing the --maxWorkers=50% option will make the tests faster in most cases. For watch mode, use --maxWorkers=25%, and for CI disable Jest workers with --runInBand. You can experiment with the percentage and fine-tune for your particular setup.
// package.json
{
"scripts": {
// standalone Jest
"test": "jest --maxWorkers=50%",
"test:watch": "jest --watch --maxWorkers=25%",
"test:ci": "jest --runInBand",
// or with Create React App
"test": "react-scripts test --watchAll=false --maxWorkers=50%",
"test:watch": "react-scripts test --maxWorkers=25%",
"test:ci": "react-scripts test --watchAll=false --runInBand"
}
}
Update 2021-03-29
While a lot of people have reported great results, I have seen some indication that on older Intel CPUs without hyperthreading the above setting results in a performance degradation. You should benchmark and validate for your particular setup.
How Jest selects the number of workers to use
The Jest test runner—that is also supplied by default with Create React App—does not run optimally out of the box.
By default, Jest will run on all available CPU threads, using one thread for the cli process and the rest for test workers. When in watch mode, it will use half the available CPU threads.
This however results in sub-optimal performance on all systems I tested on.
We can adjust --maxWorkers by either providing a number of threads, or a percentage of the available system threads. I prefer using percentage, as it's usually easy to find a value that works across multiple systems with different CPUs.
Benchmarking Jest with --maxWorkers=50%
These are the stats for the testsuite used. It's a React app with mostly unit tests:
Test Suites: 43 passed, 43 total
Tests: 1 skipped, 258 passed, 259 total
Snapshots: 2 passed, 2 total
Here are the results on an Intel i9-9900KS (5GHz / 8 cores 16 threads):
A 21% speedup.
$ hyperfine 'npm test' 'npm test -- --maxWorkers=50%'
Benchmark #1: npm test
Time (mean ± σ): 4.763 s ± 0.098 s [User: 49.334 s, System: 5.996 s]
Range (min … max): 4.651 s … 4.931 s 10 runs
Benchmark #2: npm test -- --maxWorkers=50%
Time (mean ± σ): 3.925 s ± 0.044 s [User: 27.776 s, System: 4.028 s]
Range (min … max): 3.858 s … 3.973 s 10 runs
Summary
'npm test -- --maxWorkers=50%' ran
1.21 ± 0.03 times faster than 'npm test'
And here are the results on a 2016 13" MacBook Pro (3.3GHz / 2 cores 4 threads):
A 14% speedup.
$ hyperfine 'npm test' 'npm test -- --maxWorkers=50%'
Benchmark #1: npm test
Time (mean ± σ): 14.380 s ± 0.230 s [User: 22.869 s, System: 3.689 s]
Range (min … max): 14.049 s … 14.807 s 10 runs
Benchmark #2: npm test -- --maxWorkers=50%
Time (mean ± σ): 12.567 s ± 0.213 s [User: 19.628 s, System: 3.290 s]
Range (min … max): 12.258 s … 12.942 s 10 runs
Summary
'npm test -- --maxWorkers=50%' ran
1.14 ± 0.03 times faster than 'npm test'
And finally, a 2020 M1 MacBook Air:
A 12% speedup.
$ hyperfine 'npm test' 'npm test -- --maxWorkers=50%'
Benchmark #7: npm run test
Time (mean ± σ): 5.833 s ± 0.025 s [User: 30.257 s, System: 6.995 s]
Range (min … max): 5.813 s … 5.861 s 3 runs
Benchmark #4: npm test -- --maxWorkers=50%
Time (mean ± σ): 5.216 s ± 0.060 s [User: 19.301 s, System: 3.523 s]
Range (min … max): 5.179 s … 5.285 s 3 runs
Summary
'npm test -- --maxWorkers=50%' ran
1.12 ± 0.01 times faster than 'npm test'
What about running alongside other programs?
Measuring this is harder, but I have noticed that running with --maxWorkers=25% performs the best for my use cases.
This gives the best performance for test:watch alongside code watch/hot reloading, and for running husky commit hooks in parallel.
What about CI?
In my and other's experience, --runInBand can be the fastest option for CI runs.
What does --runInBand do? From the official docs:
Run all tests serially in the current process, rather than creating a worker pool of child processes that run tests. This can be useful for debugging.
Turns out, it's also useful in resource-constrained environments like CI, where the overhead of worker processes is higher than the speedup of running tests in parallel.
Finding the optimal number of threads for a given testsuite/system
It's easy to write a small script to find the optimal number of threads for your particular usecase:
export MAX_WORKERS=15; hyperfine --parameter-scan num_threads 1 $MAX_WORKERS 'npm run test -- --maxWorkers={num_threads}' -m 3 -w 1
Here are the results on an Intel i9-9900KS (5GHz / 8 cores 16 threads):
Summary
'npm run test:jest -- --maxWorkers=7' ran
1.01 ± 0.01 times faster than 'npm run test:jest -- --maxWorkers=8'
1.02 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=6'
1.04 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=5'
1.05 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=9'
1.08 ± 0.03 times faster than 'npm run test:jest -- --maxWorkers=10'
1.11 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=11'
1.11 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=4'
1.18 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=13'
1.19 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=14'
1.21 ± 0.04 times faster than 'npm run test:jest -- --maxWorkers=12'
1.23 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=15'
1.25 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=3'
1.58 ± 0.02 times faster than 'npm run test:jest -- --maxWorkers=2'
2.55 ± 0.04 times faster than 'npm run test:jest -- --maxWorkers=1'
As you can see, the optimal number of workers in this case is 7, not the 8 that 50% would give us. However the difference between the two is within the margin of error, and 50% is more flexible.
Conclusion
Jest performance out of the box can be easily improved by tweaking maxWorkers. If you decide to test this for yourself, hyperfine makes it very easy.
Hope this was helpful! Feel free to reach out to me on Twitter @VanTanev.
Happy hacking!
Top comments (15)
I tried running our CI tests with:
--maxWorkers=50%: got worse from 13:30min to 15min--runInBand: got worse from 13:30min to 22minMaybe I'm doing something wrong. Any insights?
Thanks for sharing, I didn't expect such results! Didn't know about hyperfine either, that's an instant download for me
I was looking at this for local development, and was surprised to see max-workers at 75% boosted a slow-running test from 30s to 10s. But then I put it back at 50% and it was still 10s, same without the option anyway. So that makes me wonder, why does jest speed up locally after a few runs? Even with no cache?
Because of local filesystem caches - instead of having to hit the hard drive, the OS will store file metadata (or sometimes whole files) in memory after they have been accessed.
To reliably measure performance, you must use something like hyperfine:
hyperfine 'npm test' -w 3 // Will first run 3 warmup runs to make sure caches are primed, and only then do timing runs.
Thanks for the article, how did you benchmark --watch?
You don't need to benchmark "--watch" as it's basically running your tests over and over again.
If you want to benchmark npm script, which uses watch, just pass
--watchAll=falseto disable watch for that scriptSure, I mean if that's adequate to measure runtime performance of watched tests I 'm fine.
What a great article.
Thank you for all the insights and demonstration of this
hyperfinetool.This article is a godsend
This is really useful. My machine was extremely slow while running the jest tests, but now it's much better and my tests are also running faster.
neat, thanks for the benchmarking, definitely speeds up my tests locally
Super helpful article thank you.
Very useful, thank you! I liked the examples of configuration with CRA or with Jest alone. I saved ~20 seconds on a 65 seconds local test run, which is very appreciated. Will now test the CI version!