How I Fixed The Unexpected Token Error In Jest

I recently ran into a nasty problem that took the better part of a day to get straight. So I'm gonna put my solution here in the hopes that it helps someone else.

The Problem

I created an NPM package that uses modern JavaScript. By "modern", I mean ES2015-compliant JavaScript (which honestly doesn't feel all that... modern to me, but NPM & Jest seem to be stuck in the 2013 glory years of CommonJS - so, whatever...). I refuse to write my packages with old-skool require() and module.export and all those other aging conventions. It's 2021, dammit. Babel isn't some cutting-edge technology. I should be able to write my packages in a way that's consistent with the rest of the code in my apps.

Because I was writing this package for public/distributed consumption, I felt it was important to have good unit tests on it. And as a "React guy", I tend to default to Jest. But usually, when I use Jest, I'm testing my own little batch of code in my own self-preserved runtime environment. And when I do that, Jest works just fine.

But this time, I'm testing my own NPM package, which imports some of my other NPM packages. To put this in different terms, I'm using Jest to test a package with "modern" JavaScript, which in-turn imports another package with "modern" JavaScript. And Jest doesn't like it. Not one bit.

The "issue" is that Jest only wants to process CommonJS-style code. So for the Jest tests to run, it first needs to be transpiled by Babel. If you don't get it properly transpiled, you'll see an error like this:

Jest encountered an unexpected token

Depending upon your setup, you might see the error on the first line of the code file or you might see it when the code tries to process JSX. I saw it on line 1, because line 1 is almost always occupied by an import statement - and there are no import statements in CommonJS.

The Headache

If you Google "jest unexpected token", there are several signs that this is a really nasty issue:

1. There are a great many threads on the issue - on Stack Overflow and otherwise.
   

2. The threads span a number of years - meaning that the issue keeps cropping up for people repeatedly.
   

3. Many of the threads are long. This isn't one of those issues where a quick answer solves the problem for the original poster.
   

4. It's clear from reading through these threads that it's not your typical noob issue. Some of the people posting their approaches seem quite knowledgeable on all aspects of configuration for Jest / React / Babel / TypeScript / etc.
   

5. There doesn't seem to be any one universal answer. The threads are filled with one person posting something like, "Here's how I fixed it." - followed by several other people saying that they did the exact same thing... and it did not solve their issue.
   

6. The proposed answers all seem to be quite environment-specific. Sometimes you need to use transformIgnorePatters - but on other builds, that does nothing. Working on Windows? You'll probably need cross-env somewhere in your solution. Or maybe win-node-env. Or maybe env-cmd. Or maybe windows-environment. If you're in React, you'll probably need a different solution than Vue. And both of those solutions could be different if you're using TypeScript. You'll probably need a properly-configured .babelrc file - but maybe you'll need to change that to babel.config.json?
   

FWIW, I even found several articles right here on Dev.to with proposed solutions - that did nothing for me.

Before I get into my solution, I just gotta say that, IMHO, Babel and/or Jest have a real problem here. When you see this many people struggling over something for this long - people who otherwise seem to know what they're doing - well... something really needs to be optimized in this process.

Disclaimer

If you haven't figured out already, this whole Babel / WebPack / Jest / React configuration thing confuses me sometimes. And yes, this is even coming from a guy who's been doing this stuff very heavily for decades. Some guys really get off on solving these types of problems - but they just annoy me. I end up spending sooooo much time wrestling with an issue that I honestly don't care that much about, and it's just keeping me from coding.

With that in mind, I absolutely do NOT know how to solve this for every configuration - or even most of them. I just know what I finally got to work. So this article might be as useless to you as all the others that I cycled through in the last few days. But hopefully it will save someone a little time.

As I already mentioned, these solutions seem to be very environment specific. So you should probably know that I'm working on a Windows 10 machine with Node v14.2.0, NPM v6.14.4, and Jest v26.6.3 installed locally.

Solution #1 - A Standalone JS Project

package.json (abridged)

{
  "name": "@toolz/allow",
  "main": "src/allow.js",
  "scripts": {
    "test": "jest --transformIgnorePatterns \"node_modules/(?!@toolz/allow)/\" --env=jsdom"
  },
  "type": "module",
  "devDependencies": {
    "@babel/cli": "^7.13.0",
    "@babel/core": "^7.13.1",
    "@babel/node": "^7.10.5",
    "@babel/plugin-transform-modules-commonjs": "^7.13.0",
    "@babel/preset-env": "^7.11.0",
    "@babel/preset-react": "^7.12.13",
    "babel-jest": "^26.6.3",
    "babel-plugin-transform-class-properties": "^6.24.1",
    "babel-preset-jest": "^26.6.2",
    "jest": "^26.6.3",
    "jest-cli": "^26.6.3",
  },
  "dependencies": {
    "@toolz/is-a-regular-object": "^1.0.1"
  }
}

Pay particular attention to the scripts: test node. The name of this project (@toolz/allow) is in the parentheses. Also, this did not work until I set the env value to jsdom. I don't really think I need all of that stuff in the devDependencies node. But you know what?? It works right now - so I ain't touchin it.

babel.config.json

{
  "presets": [
    "@babel/preset-env"
  ]
}

NOTE: This is not .babelrc. In this particular setup, I appear to have needed the file to be babel.config.json.

With these settings, I can now run npm test and it properly runs my tests - including those that require an import of @toolz/is-a-regular-object.

Solution #2 - A React Project (with create-react-app)

package.json (abridged)

{
  "name": "@toolz/allow-react",
  "dependencies": {
    "@testing-library/jest-dom": "^5.11.9",
    "@testing-library/react": "^11.2.5",
    "@testing-library/user-event": "^12.7.2",
    "@toolz/allow": "^1.0.1",
    "@toolz/is-a-regular-object-react": "^1.0.0",
    "react": "^17.0.1",
    "react-dom": "^17.0.1",
    "react-scripts": "4.0.2",
    "web-vitals": "^1.1.0"
  },
  "scripts": {
    "test": "react-scripts test --transformIgnorePatterns \"node_modules/(?!@toolz/allow-react)/\" --env=jsdom"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "devDependencies": {
    "@babel/plugin-transform-modules-commonjs": "^7.12.13",
    "babel-jest": "^26.6.3"
  }
}

Consistent with create-react-app applications, there's no .babelrc or babel.config.json file in this project. Everything I need is right here in package.json. This now runs all tests with npm test, including those that import from other ES2015-syntax projects.

As I've tried to make painfully clear, I have no idea if this will work in your project. Heck, it probably won't. But maybe these configs will help someone?