DEV Community

Cover image for Building a MongoDB migration system for NestJS with mongoose
Julien Prugne for Webeleon

Posted on • Updated on

Building a MongoDB migration system for NestJS with mongoose

This tutorial is assuming:

  • You have a working NestJS project
  • You are using MongoDB and mongoose

In the recent past, I had an urge to do a bit of refactoring on my discord game.
Still a work in progress, but I couldn't stand anymore the fact that houses were named homes...
I already can hear you, 'just change labels displayed no one care!'.

FAUX

I do care about the consistency of naming in my codebases. If homes are houses then the next thing you know is:
Canons will become wooden swords and wolves are dogs...

I spent some time online looking for solutions and I finally built something I like.
Let me present to you the result of my work.

I chose to use the migrate library since it is database agnostic, offers an easy up/down logic, and can store the migration status in any form.

Enough speaking about me, let me guide you through this journey.

Install migrate

Go on install the bad guy!



npm i --save migrate


Enter fullscreen mode Exit fullscreen mode

Create a folder to store your migrations!

You will new two folders:



mkdir src/migrations
mkdir src/migrations-utils


Enter fullscreen mode Exit fullscreen mode

The first one will store the update scripts and the seconds will store some utilities.
Let's look into the seconds.

Some little helpers

In the introduction, I told you that migrate is database agnostic.
Therefore you need to write a little mongodb connector:



import { MongoClient } from 'mongodb';
import { configs } from '../config/configuration';

const MONGO_URL = configs.mongoUrl;

export const getDb = async () => {
  const client: any = await MongoClient.connect(MONGO_URL, { useUnifiedTopology: true });
  return client.db();
};


Enter fullscreen mode Exit fullscreen mode

Nice! let's keep going.

antilope break

Migrate is a tool made in javascript.
And we use typescript, the best thing to do is have a little template with the database already connected.



import { getDb } from '../migrations-utils/db';

export const up = async () => {
  const db = await getDb();
  /*
      Code your update script here!
   */
};

export const down = async () => {
  const db = await getDb();
  /*
      Code you downgrade script here!
   */
};


Enter fullscreen mode Exit fullscreen mode

I had some trouble with ts-node/register in migrate command line.
This little helper solved my transpilation errors!
Do the same! now! do it!



// eslint-disable-next-line @typescript-eslint/no-var-requires
const tsNode = require('ts-node');
module.exports = tsNode.register;


Enter fullscreen mode Exit fullscreen mode

Update package.json

it's time for you to update the package.json of your project in order to have easy to use scripts for the future!

A script to generate migration files

Add this sweet line in the package.json, in the script section!



"migrate:create": "migrate create --template-file ./src/migrations-utils/template.ts --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\"",


Enter fullscreen mode Exit fullscreen mode

--template-file ./src/migrations-utils/template.ts provide a template file, it's a necessary thing since we are in a typescript repo.
It also provides you an easy way to bootstrap migration just the way you like it!

--migrations-dir=\"./src/migrations\" Tell migrate where your migration scripts are stored.
By default, it's at the project root.

--compiler=\"ts:./src/migrations-utils/ts-compiler.js\" Explain to migrate how to handle typescript files.

Now, you just need to run this command to create an empty typescript migration file in the correct folder!



npm run migrate:create -- <migration name>


Enter fullscreen mode Exit fullscreen mode

A script for upgrades and a script for downgrades

AAAAAAnd two more lines in the package.json, again in the scripts section!



"migrate:up": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" up",
"migrate:down": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" down"


Enter fullscreen mode Exit fullscreen mode

No new options here, I already explained them but refreshing is nice.

--migrations-dir=\"./src/migrations\" Tells migrate where to find your migrations!

--compiler=\"ts:./src/migrations-utils/ts-compiler.js\" Tells migrate how to handle typescript...

You can now run update script: npm run migrate:up or downgrade script npm run migrate:down

What will happen when you run a migration?

Migrate will store your migration state in a file at the project root.
This file is called migrate.json.
It looks like this:



{
  "lastRun": "1605197159478-test.ts",
  "migrations": [
    {
      "title": "1605197159478-test.ts",
      "timestamp": 1605197181474
    }
  ]
}


Enter fullscreen mode Exit fullscreen mode

DO NOT COMMIT migrate.json

Questions?

questions

I'll be glad to answers questions in the comments.

If you liked my discord consider joining my coding lair!
☎️Webeleon coding lair on discord

You can also email me and offer me a contract 💰
✉️Email me!

And since I'm a nice guy, here, take this sample repo containing a working codebase!
🎁Get the code of the tuto from github

Buy Me A Coffee

Documentation

documentation

Top comments (7)

Collapse
 
jdnichollsc profile image
J.D Nicholls

Thanks for sharing!

I was wondering how to apply database migrations with Mongoose in a NestJS template but I figure out that I was trying to implement a Database seeding instead because you don't need migrations with a NoSQL Database. This is my template: github.com/proyecto26/MyAPI/tree/n...

Best
Juan

Collapse
 
bassochette profile image
Julien Prugne

In production database you might need to apply migration on existing data even in document oriented database.
For example: you can clean some legacy data. (GDPR compliance for example)
Seeding is only for new data or starting a new environment.

Collapse
 
buryo profile image
Bsn

That's true, I see the use of it now thanks!

Collapse
 
mohsenheydari profile image
Mohsen Heydari • Edited

Thank you for sharing Julien!
I had a problem with relative paths in tsconfig.json and I managed to solve the issue by using the following code in 'ts-compiler.js':

const tsNode = require('ts-node');
const tsConfigPaths = require('tsconfig-paths');

module.exports = function(opts) {
    tsConfigPaths.register();
    return tsNode.register(opts);
};
Enter fullscreen mode Exit fullscreen mode
Collapse
 
pratik149 profile image
Pratik Rane

Awesome! Thanks for this Julien. :)

Collapse
 
cneri profile image
c-neri

Thank you for sharing!
I have a question, where should i put this code?
const tsNode = require('ts-node');
module.exports = tsNode.register;

Thank you in advance!

Collapse
 
bassochette profile image
Julien Prugne

Hello,

I use to place it in a file called ts-compiler.js in a folder called src/migration-utils
You can take a look in the sample repo.

Then you can use as an option in the migration command invocations: --compiler=\"ts:./src/migrations-utils/ts-compiler.js\"

It's in the sample repo, but I'll share the migration commands in the package.json here:

"migrate:create": "migrate create --template-file ./src/migrations-utils/template.ts --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\"",
"migrate:up": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" up",
"migrate:down": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" down"
Enter fullscreen mode Exit fullscreen mode

Hope this nugget will help you :)