Maintainable, readable, DRY JavaScript unit tests without "magical" sinon, proxyquire, jest, etc.
Disclaimer
This post inspired by @samueldjones article called
Thanks Sam!
Intro
Let me start by saying that, as a professional, I found testing to be pretty simple and easy. I created a number of apps that made millions for my employers and all with the help of test coverage! However, having spent the last years working across the full stack and writing a lot of unit and integration tests in the process, I feel urge to share my vast experience. While it does require a slight change of perspective, testing should feel like taking a gentle stroll through the countryside away from the hustle and bustle of business logic.
Practical
Diving straight into our scenario then. We simply want to test our function that fetches all available currency rates from this URL - https://currencylayer.com/, then stores all of them to MongoDB database for historical purposes and caches latest of them to Redis database, but only if your node.js process have access to the databases. A use case for this API could potentially be a serverless function that runs daily (hourly?) to fetch all exchange rates for that day in order to show a historical chart of currency conversion rates and have latest exchange rate cached in an in-memory database (Redis).
"Wow!", you might think, "This is a lot to mock!". Indeed a lot. I intentionally made Sam's original example much more complex to get closer to real world scenarios.
Let's pause here to review the packages that we will be using:
- stampit: flexible object factories
That's all! Nothing else. No other dependencies need. Yay!!!
Please note, we are not using proxyquire and sinon (and even chai). Why? Because years of my experience drove we away from the approaches these libraries take. They incur too much JavaScript "magic". The more "magic" your code has, the less maintainable it is, the more problems you'll have upgrading your node_modules. These problems cumulative effect accounted for up to 20% of my work time.
At the end I came to conclusion that "magic" is bad and the more explicit your code - the better.
My general recommendations for maintainable node.js code:
- Less magic, more explicit.
- Fewer dependencies.
- More simplicity.
- Less code. And less auto generated code.
Our main code
// ExRateFetcher.js
const CURRENCY_API_URL = "https://api.currencylayer.com";
const ACCESS_KEY = process.env.ACCESS_KEY;
module.exports = require("stampit")({
name: "ExRateFetcher",
props: {
fetch: global.fetch,
mongoose: require("mongoose"),
CurrencyPairModel: null,
redis: require("redis"),
redisClient: null,
},
init() {
const client = this.redis.createClient(process.env.REDIS_URL);
client.on('ready', () => {
this.redisClient = client;
});
this.mongoose.connect(process.env.MONGO_URL)
.then(() => {
const CurrencyPairSchema = new this.mongoose.Schema({
_id: String, // currency pair as primary key
rates: [{ date: String, rate: Number }]
});
this.CurrencyPairModel = this.mongoose.model(
'CurrencyPair',
CurrencyPairSchema
);
});
},
methods: {
async _saveToMongo(rates, date) {
const date = date.toISOString().substr(0, 10);
for (const [pair, rate] of rates) {
await this.CurrencyPairModel.upsert(
{ _id: pair, "rates.date": date },
{ $set: { rate } }
);
}
},
async _saveToRedis(rates) {
for (const [pair, rate] of rates) {
await this.redisClient.set(pair, rate);
}
},
async fetchAndStoreLatest() {
const responseBody = await this.fetch(`${CURRENCY_API_URL}/live?access_key=${ACCESS_KEY}`);
const date = new Date(responseBody.timestamp * 1000);
const rates = Object.entries(responseBody.quotes);
if (this.CurrencyPairModel) {
await this._saveToMongo(rates, date);
}
if (this.redisClient) {
await this._saveToRedis(rates);
}
}
}
});
Here is the usage of ExRateFetcher.js:
const ExRateFetcher = require("./ExRateFetcher.js");
ExRateFetcher().fetchAndStoreLatest();
I/O dependencies
Some APIs might be a giant Java Spring server. Some APIs might be too dangerous to call (e.g. some of the AWS APIs). Some APIs can be too expensive (e.g. Authy). Some databases can't be easily rolled out for unit testing purposes (e.g. Kafka). Some I/O can be a third party gRPC, UDP, or WebSocket server. You can't use any of these on a CI or your laptop.
In real world the third party APIs and databases you connect to might not be available in your CI/CD environment. In my experience, about half of the I/O dependencies (APIs, DBs, etc) are typically impossible to have for unit testing purposes. Thus...
INABILITY TO MOCK IS A CODE SMELL in node.js.
Our unit test
const assert = require("assert");
const { describe, it } = require("node:test");
// Let's stub two database dependencies with no-op code.
const ExRateFetcher = require("./ExRateFetcher").props({
// Attention! Mocking redis!
redis: { createClient: () => ({ on() {} }) },
// Attention! Mocking mongoose!
mongoose: { connect: () => ({ then() {} }) },
});
describe("ExRateFetcher", () => {
describe("#fetchAndStoreLatest", () => {
it("should fetch", (done) => {
const MockedFetcher = ExRateFetcher.props({
// Attention! Mocking global fetch!
async fetch(uri) {
assert(uri.includes("/live?access_key="));
done();
}
});
MockedFetcher().fetchAndStoreLatest();
});
const responseBody = {
"timestamp": 1432400348,
"quotes": {
"USDEUR": 1.278342,
"USDGBP": 0.908019,
}
};
it("should store in Redis", () => {
let redisSetCalled = 0;
const MockedFetcher = ExRateFetcher.props({
fetch: async () => responseBody,
// Attention! Mocking redis!
redis: {
createClient() {
return {
on(event, callback) {
assert(event === "ready");
assert(typeof callback === "function");
setTimeout(callback, 0);
},
async set(key, value) { // DB call mocking
assert(responseBody.quotes[key] === value);
redisSetCalled += 1;
}
};
}
},
});
const fetcher = MockedFetcher();
await new Promise(r => setTimeout(r, 1)); // wait connection
await fetcher.fetchAndStoreLatest();
assert(redisSetCalled === 2);
});
it("should store in MongoDB", () => {
let mongoUpsertCalled = 0;
const MockedFetcher = ExRateFetcher.props({
fetch: async () => responseBody,
// Attention! Mocking mongoose!
mongoose: {
connect() {
return {
then(callback) {
assert(typeof callback === "function");
setTimeout(callback, 0);
}
};
},
Schema: function () {},
model: () => ({
async upsert(query, command) { // DB call mocking
assert(command.$set.rate === responseBody.quotes[query._id]);
assert(query["rates.date"] === "2015-05-23");
mongoUpsertCalled += 1;
}
}),
},
});
const fetcher = MockedFetcher();
await new Promise(r => setTimeout(r, 1)); // wait connection
await fetcher.fetchAndStoreLatest();
assert(mongoUpsertCalled === 2);
});
});
});
When I see sinon in a codebase, there is usually a lot of repetitive mocking happening.
- Test 1 - mock A, mock B, mock C
- Test 2 - mock A, mock B, mock C
- Test 3 - mock A, mock B, mock C
Whereas, in the unit test code above we mock only the bare minimum. We do not over mock things. Also, you don't need to go to sinon docs over and over again to remember what is the syntax for, say, event emitter or mongoose model object.
In my experience, the code above is very much stable and quite flexible to mock literally anything.
I often mock setTimeout or other JavaScript/Node.js globals. Whereas, mocking globals in node.js is so very much error prone and unstable if using proxyquire, sinon, jest, etc. Using the approach above you could mock setTimeout only in that specific test and nowhere else. This trick alone saved me days over the years.
See more about stampit module here: https://stampit.js.org/
Top comments (0)