DEV Community

Cover image for Setup in-memory database for testing Node.js and Mongoose
Dmytro Rykhlyk
Dmytro Rykhlyk

Posted on • Edited on

Setup in-memory database for testing Node.js and Mongoose

#node #mongoose #jest #supertest

I've been working on creating an application using Node.js and Mongoose where all data stored in the cloud MongoDB Atlas. My goal was to test API endpoints and write some unit tests, for that I found this package called mongodb-memory-server, which allows me to create a connection to the MongoDB server from my cache folder without a need to use cloud test database or installing mongodb client.

NOTE: Full version of project's code for mongodb-memory-server's new version @7.2.1 - Github repo. (old version @6.9.3 - Github repo)

📝 Prepare sample project

In this example project, we'll create a mongoose model and add services to execute some operations with the database. In addition, we'll add some basic user authentication. Here's how a complete project structure will look like:

├── models
│   ├── User.js
│   └── Post.js
├── middlewares
│   └── auth.js
├── services
│   ├── user.js
│   └── post.js
├── tests
│   ├── db.js
│   ├── auth.test.js
│   └── post.test.js
├── app.js  
├── server.js
├── package.json
├── README.md
└── ...
Enter fullscreen mode Exit fullscreen mode

We'll use the following dependencies that can be installed using npm:

npm i mongoose express nodemon dotenv jsonwebtoken cookie-parser
Enter fullscreen mode Exit fullscreen mode

Note: We'll execute app.listen(port) in the server.js file to be able to export app.js as an agent to supertest module. That's why you should also check not to create a connection to an actual database when running tests.

// app.js

const express = require('express');
const mongoose = require('mongoose');

require('dotenv').config();
const { MONGO_DB_CONNECT } = process.env;

const app = express();

// NOTE: when exporting app.js as agent for supertest 
// we should exlcude connecting to the real database
if (process.env.NODE_ENV !== 'test') {
  mongoose.connect(MONGO_DB_CONNECT, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  });
}
Enter fullscreen mode Exit fullscreen mode

✨ Setup dependencies and configure jest

We'll install some packages: 

npm install --save-dev jest mongodb-memory-server supertest
Enter fullscreen mode Exit fullscreen mode

First, we'll add jest configuration in the package.json:

// package.json

"jest": {
    "testEnvironment": "node"
  },
 "scripts": {
    "test": "jest --watchAll --coverage --verbose --silent --runInBand"
  }
Enter fullscreen mode Exit fullscreen mode

Note: Adding --runInBand parameter will make all tests run serially to make sure there's only one mongod server running at once.

✨ Setup In-memory database

Take database handle function from the mongodb-memory-server official documentation to than start server in each test file.
Check original: Simple Jest test example

NOTE: I've got an error: MongooseError: Can't call openUri() on an active connection with different connection strings and to solve this problem I close previous connection before connecting to the new one.

// tests/db.js

const mongoose = require('mongoose');
const { MongoMemoryServer } = require('mongodb-memory-server');

let mongoServer;

// For mongodb-memory-server's old version (< 7) use this instead:
// const mongoServer = new MongoMemoryServer();

const opts = {
  useNewUrlParser: true,
  useUnifiedTopology: true,
};

// Provide connection to a new in-memory database server.
const connect = async () => {
  // NOTE: before establishing a new connection close previous
  await mongoose.disconnect();

  mongoServer = await MongoMemoryServer.create();

  const mongoUri = await mongoServer.getUri();
  await mongoose.connect(mongoUri, opts, err => {
    if (err) {
      console.error(err);
    }
  });
};

// Remove and close the database and server.
const close = async () => {
  await mongoose.disconnect();
  await mongoServer.stop();
};

// Remove all data from collections
const clear = async () => {
  const collections = mongoose.connection.collections;

  for (const key in collections) {
    await collections[key].deleteMany();
  }
};

module.exports = {
  connect,
  close,
  clear,
};
Enter fullscreen mode Exit fullscreen mode

✨ Write tests

Now each test file should contain the same code at the top.

  • Connect to a new in-memory database before running any tests.
  • Remove all test data after every test. (optional)
  • After all tests - remove and close the database and server. 
// tests/post.test.js

const request = require('supertest');
const app = require('../app');
const db = require('./db');

// Pass supertest agent for each test
const agent = request.agent(app);

// Setup connection to the database
beforeAll(async () => await db.connect());
beforeEach(async () => await db.clear());
afterAll(async () => await db.close());


describe('test smthing', () => {
  test('It should do smthing',  done => {
     // some tests
  });
});
Enter fullscreen mode Exit fullscreen mode

✔ Some examples of tests

Here's a list of some tests:

1. Store data to the database 

// tests/post.test.js
// ...

describe('POST /api/posts/create', () => {
  test('It should store a new post',  done => {
    agent
      .post('/api/posts/create')
      .send({ title: 'Some Title', description: 'Some Description' })
      .expect(201)
      .then(res => {
        expect(res.body._id).toBeTruthy();
        done();
      });
  });
});
Enter fullscreen mode Exit fullscreen mode

2. Test that a service function doesn't throw an error 

// tests/post.test.js
// ...

const { create } = require('../services/post');

describe('services/post.js', () => {
  test('It should return a post with an id', done => {
    expect(async () => create({ title: 'Some Title', description: 'Some Description' })).not.toThrow();
    done();
  });
});
Enter fullscreen mode Exit fullscreen mode

3. Test protected routes using JWT token in cookies 

// tests/auth.test.js
// ...

describe('POST /api/user/signup', () => {
  test('It should return protected page if token is correct',  async done => {
    let Cookies;

    // Create a new user
    await agent
      .post('/api/user/signup')
      .send({ email: 'hello@world.com', password: '123456' })
      .expect(201)
      .then(res => {
        expect(res.body.user).toBeTruthy();

        // Save the cookie to use it later to retrieve the session
        Cookies = res.headers['set-cookie'].pop().split(';')[0];
      });

    const req = agent.get('/');
    req.cookies = Cookies;

    req.end(function(err, res) {
      if (err) {return done(err);}

      expect(res.text).toBe('Protected page');
      expect(res.status).toBe(200);
      done();
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

NOTE: Full version of project's code for mongodb-memory-server new version @7.2.1 - Github repo. (old version @6.9.3 - Github repo)

Links

Thanks for reading!

P.S.: 🔰 This is my first blog post.
This article is just a practical guide to setup some basic testing environment. Use links to learn more about these tools.

Top comments (15)

Collapse
 
tperrinweembi profile image
tperrin •

Thanks for this awsome article. But since mongodb-memory-server version 7.0.0 we can't use anymore const mongoServer = new MongoMemoryServer(); but must use instead const mongo = await MongoMemoryServer.create(); . But I didn't succeed to adapt this to your code since that make me run await at file top level so I got the error "SyntaxError: await is only valid in async function". How would do you migrate your code to version 7?

Collapse
 
ryuuto829 profile image
Dmytro Rykhlyk •

Thank you for your comment!
I've updated article using the latest @7.2.1 version of mongodb-memory-server.
Basically, I define a new instance of "MongoMemoryServer" in the connect async function to automatically start server:

/tests/db.js

- const mongoServer = new MongoMemoryServer();
+ let mongoServer;

const connect = async () => {
  await mongoose.disconnect();
+ mongoServer = await MongoMemoryServer.create();

// ...
};
Enter fullscreen mode Exit fullscreen mode
Collapse
 
tperrinweembi profile image
tperrin •

Ok! thanks a lot for your answer

Collapse
 
tperrinweembi profile image
tperrin •

Simply move the create() into the connect method. Thanks! (answer from this article : dev.to/paulasantamaria/testing-nod...)

Collapse
 
nicolasbellini profile image
nicolasbellini •

Hello, thanks for your guide.
How do i test something that requires a JWK, since i first need to create something in the DB for it to provide a token and then use that token for the update request

Collapse
 
ryuuto829 profile image
Dmytro Rykhlyk • • Edited

Hi, thank you :)
Its a good question! Please, check my updated Github repo where I added a simple authentication using JWT and some tests.

My solution is based on this issue

// auth.test.js

describe('POST /api/user/signup', () => {
  test('It should return protected page if token is correct',  async done => {
  // Store our cookies
    let Cookies;

    // Create a new user
    await agent
      .post('/api/user/signup')
      .send({ email: 'hello@world.com', password: '123456' })
      .expect(201)
      .then(res => {
        expect(res.body.user).toBeTruthy();

        // Save the cookie to use it later to retrieve the session
        Cookies = res.headers['set-cookie'].pop().split(';')[0];
      });

    const req = agent.get('/');
    req.cookies = Cookies;

    req.end(function(err, res) {
      if (err) {return done(err);}

      expect(res.text).toBe('Protected page');
      expect(res.status).toBe(200);
      done();
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

You can create a user and generate a token in each test or just populate it through beforeAll:

beforeAll(async () => {
  await db.connect();
  // create user
});
Enter fullscreen mode Exit fullscreen mode
Collapse
 
parveen99 profile image
parveen99 •

Hi great article. My project is also very similar. How can I learn how to write unit tests using mongo in memory server. Its my first time writing unit tests as a developer.

Collapse
 
ryuuto829 profile image
Dmytro Rykhlyk • • Edited

Thank you!
It's hard for me to give you some concrete advice, I'm still learning how to write tests :) For my small project though, I've tried to write tests for each API endpoint using superset, Jest and mongodb-memory-server official documentations along with some examples.

Collapse
 
ocholla_t profile image
Ocholla-T •

Created a dev.to account just to heart this article.

Collapse
 
ryuuto829 profile image
Dmytro Rykhlyk •

Thanks a lot! 😉

Collapse
 
cedricgourville profile image
Cédric Gourville •

Hey thx for the article. In the function clear from the db file in test. I think deleMany should have an empty object as parameter

Collapse
 
ryuuto829 profile image
Dmytro Rykhlyk • • Edited

Hi, thank you!
I've tested it with / without empty object and in both cases, all data was properly deleted after each test, so everything should work fine.
Anyway, I'll add it as in the MongoDB documentation. Thx :)

Collapse
 
kamuzinzi profile image
Egide Kamuzinzi •

I have followed your post but still getting errors on my project.

ReferenceError: regeneratorRuntime is not defined
Enter fullscreen mode Exit fullscreen mode

Any help?

Collapse
 
navyaharish profile image
navyaharish •

Do you have any similar repo with examples for nestJs testing using mongoMemoryServer?

Collapse
 
ryuuto829 profile image
Dmytro Rykhlyk •

Sorry, I don't have examples for nestJs 😔

View full discussion (15 comments)

Read next

mayallo profile image

How I Improved Video Streaming with FFmpeg and Node.js

Mohamed Mayallo -

hisukurifu profile image

Running local LLM (Ollama) in your nodejs project.

Aniket Dhakane -

diegoleteliers10 profile image

Making your First Project with Deno 2.0

Diego Letelier -

ken2026 profile image

Crafting a Scalable Node.js API: Insights from My RealWorld Project with Express, Knex, and AWS CDK

Ken Yip -