DEV Community

Cover image for From Jest to Vitest - Migration and Benchmark
Matti Bar-Zeev
Matti Bar-Zeev

Posted on

From Jest to Vitest - Migration and Benchmark

In this post join me as I migrate my project’s test runner framework from Jest to Vitest, and check if it is really as they claim - “A blazing fast unit test framework”.

Vitest?

I’m pretty sure you’ve already heard about Vite which is Evan You’s recent promising project for frontend tooling. Benchmarks claim it is super fast in times where bundling and HMR present real challenges for dev velocity.
It was recently brought to my attention that there is another emerging project called Vitest which is supposed to bring that same disruption into the realm of test runners. My curiosity obviously could not let this pass.
Although “Vitest is still in development and not stable yet. It's not recommended to use it in production.” I was eager to find out just how fast we are talking about.

Here's my path for migrating a project using Jest into using Vitest along with some benchmarking.

Benchmarking

My guinea pig is (yet again) my React Word-Search game and its tests, but before I jump in I would like to measure how long it currently takes to run the project’s tests in order to do some comparison at the end.
The Project currently has 4 test files which have 37 tests between them. Not too many, I agree, but I believe it can give a good sense of the difference between Jest and Vitest.

I will measure the tests running time in 2 scenarios:

  • Initial run - where I first lunch the test
  • Watch run - where I change a single file and see how long it takes for the tests to run

For each framework I will run these scenarios a few times (with and without cleaning the cache for Jest for the initial run) to make sure I get the average time.

Here are the results for Jest I got:

  • Initial run - 6.5s (without clearing Jest cache its 5.5s)
  • Watch run - 5.5s

Migrating to Vitest

Vitest has a really comprehensive gitHub repo, with some good documentation (considering its age).
I could look into the code examples for React and React Testing Lib and monkey copy-paste it with fingers crossed, but I would like to know what’s really going on, and what is the exact minimum required for one to make this transition.
Before I even start to npm install anything, I’d like to try it as they suggest by running a single test. This following test is passing with Jest, now let’s try it with Vitest:

npx vitest src/components/Confirmation/index.test.js
Enter fullscreen mode Exit fullscreen mode

After confirming the installation of ‘vitest’ we get our feedback from the test runner - who could have guessed, the first error (I must admit that Vitest has a more clearer way of displaying the errors and failed tests):

Image description

Error: Failed to parse source for import analysis because the content contains invalid JS syntax. If you are using JSX, make sure to name the file with the .jsx or .tsx extension.

While Jest has no issue with parsing these files, it appears that Vitest does not know how to parse them, and requires that we change the file names if they contain JSX.
Before I jump into introducing new configurations I would like to see if just changing the file name will help with this error, and it does - changing the name of my test file from index.test.js to index.test.jsx eliminates that error, but now I’m getting a new one:

Image description

describe is not defined

Well, Jest has these globals declared, but it seems that Vitest does not, and we need to import them explicitly. No worries, let’s do that, but before we do, we need to install Vitest. We at least know now that running Vitest just by using npx is not enough when migrating a project to work with it.

npm i vitest -D
Enter fullscreen mode Exit fullscreen mode

Now let’s add the needed imports to our test file:

import { it, describe, expect } from 'vitest';
Enter fullscreen mode Exit fullscreen mode

Oh my, now all my tests fail with a lot of errors flying, but that’s good. Let’s address them one by one:

Image description

document is not defined

This error comes from react-testing-library and it has to do with js-dom support of vitest. I’m going to look for some resources for this… yes, the docs do not fail - it says that adding a docblock or comment specifying the env as js-dom (or dom-happy) will do the trick. I will add it to my test and see how it goes:

/**
* @vitest-environment jsdom
*/

describe('Confirmation component', () => {
   . . .
Enter fullscreen mode Exit fullscreen mode

The tests run again, but still all of them are failing, now with new error:

Image description

Invalid Chai property: toBeInTheDocument

Chai? No, no, no… toBeInTheDocument is not a Chai property.
toBeInTheDocument is an API of the testing-library’s js-dom, and the part responsible to include it and append its assertions is the test setup file (in create react app it is the testSetup.js file on the project root).

In order to let vitest include this file as its own setup we need to create a vitest config, no escape there. Now is a good time to look at the configuration found on the example and check what’s going on in the configuration there. Again, I’m not blindly copy-pasting and so I pick what I know to be relevant to the problem I’m facing.

In order to use the configuration I need to install “vite”. I’m not very keen about it, but if that makes my tests run faster, so be it:

npm i vite -D
Enter fullscreen mode Exit fullscreen mode

I create a configuration file named “vite.config.js” and set the configuration as follows:

import {defineConfig} from 'vite';

export default defineConfig({
   test: {
       globals: true,
       setupFiles: 'src/setupTests.js',
   },
});
Enter fullscreen mode Exit fullscreen mode

As you can see I’m giving the setup file location, which loads the jest-dom needed, and also notice that I have the global property set to “true”. This means that I won’t need to import those global variables Jest comes with like “describe”, “expect” etc. I can remove that import from my test :)
(more information on the configuration can be found here)

Good progress, but do our tests pass now? No, some still don’t. We have another error:

Image description

jest is not defined

Well of course it isn’t. We’re using jest in this test for creating spy/stub functions with jest.fn(), but Vitest has another way of achieving this - it has the same implementation but under “vi”. So instead we need to use vi.fn()

import {vi} from 'vitest';
Enter fullscreen mode Exit fullscreen mode
it('should be able to receive a handler for the "Cancel" button and execute it upon click', () => {
      const onCancellationHandler = vi.fn();
    . . .
});
Enter fullscreen mode Exit fullscreen mode

Hurrah! We have a single test migrated into Vitest :)

I will now attempt to run the entire tests with vitest. I will start by changing my npm script for test to run vitest instead of jest:

"scripts": {
       "test": "vitest",
       . . .
},
Enter fullscreen mode Exit fullscreen mode

Let's also add the environment: 'jsdom' to the configuration file so we can avoid adding the env docblock in each test.
Running npm tests, and as you probably guessed it, many tests fail, but the good news is that there is nothing new to the issues we already bumped into before.
It is time to do some benchmarking

Benchmark again and compare

Now it is time to take our statistics again for Vitest:

  • Initial run - 5.30s (nice, but kinda the same as Jest with cache)
  • Watch run 1.25s (wow!)

Let’s put it in a nice table:

Framework Initial run Watch run
Jest 6.50s 5.5s
Vitest 5.30s 1.25s

From this little benchmarking I did here on my own machine, it appears that although the initial runs are slightly in the favor of Vitest, the watch run is a lot faster!
As I see it, there is no question that once Vitest is ready for production you should really consider replacing your current test runner with it. My Word-Search game already has it ;)

As always, if you have any thoughts or comments about what's written here, please share with the rest of us :)

Hey! If you liked what you've just read check out @mattibarzeev on Twitter 🍻

Photo by Florian Steciuk on Unsplash

Discussion (14)

Collapse
jjhiggz profile image
Jonathan Higger • Edited on

I've been using vitest because of remix stacks. Almost everything feels the same as jest but I notice a few things pragmatically.

Pros

  • nicer coloring
  • faster tests
  • less finnicky config maybe

Cons

  • it seems to crash a bit more often
  • .only and .skip don't seem to work as well for me
Collapse
elthrasher profile image
Matt Morgan

My friends, family and colleagues told me I shouldn't run off after another test framework, but now I know we're right and they're wrong. Thanks for this article!

Collapse
roblevintennis profile image
Rob Levin

I’m definitely keeping my eyes on vitest and this post makes me wanna give it another shot (forget exactly why but had trouble getting it working a few weeks back) but the whole philosophy of vitest is really compelling. Thanks for the nice case study!

Collapse
mbarzeev profile image
Matti Bar-Zeev Author

With pleasure :)

Collapse
larsejaas profile image
Lars Ejaas

Oh, that looked surprisingly simple!
I am still a bit unsure how enzyme would fit into the Vitest environment and how easy it is to setup with next.js - but the Vitest project sounds SO promising!

Collapse
robole profile image
Rob OLeary • Edited on

I hope vitest gets more traction and becomes a solid alternative to jest for vue and svelte also soon. thanks for the write-up

Collapse
luispa profile image
LuisPa

Great post!

Collapse
naeemo profile image
Naeemo

Thank you for this post, just like a "TL,DR" for me to migrate to vitest.

Collapse
mbarzeev profile image
Matti Bar-Zeev Author

Cheers :) Just remember that this is not production ready yet, according to their readme.

Collapse
tombyrer profile image
Tom Byrer

Hard to tell how much improvement there is, since we don't know how many & how complicated the tests are.

Collapse
timritzer profile image
Tim Ritzer

I mean he literally said it was 4 test files, and 37 tests. A small sample size, sure, but large enough to show relative progress.

I'd love to see some numbers for larger codebases, but that's tough since the software is still considered not production ready.

Collapse
mbarzeev profile image
Matti Bar-Zeev Author • Edited on

True. I won't use my little PoC here to determine whether or not you should use Vitest, but I do say it does look promising.

Collapse
alonseg profile image
Alon Segal

as always, love to read your articles!
their comment on not being ready for production and the tight coupling to vite will cause me to wait some more :)

Collapse
mbarzeev profile image
Matti Bar-Zeev Author

Cheers :)
Yeah, I'd wait for it some a little bit more