Multidatabase with mongoose

Introduction

At Woovi we always focus on the simplest software architecture that can support our product requirements.
Although we started using one database to support all our features, we decided to move some collections of logs to another database to reduce the load from the core database.

Adding support to multi databases

We had only a MONGO_URI environment variable.
We decided to create one environment variable for each other database we are going to add. We are following the pattern MONGO_URI_<>.

After adding the new env, we need to create a connection to this database:

// default connection
await mongoose.connect(config.MONGO_URI, options);

// other connections
await mongoose.createConnection(config.MONGO_URI_LOGS)
.asPromise();

Mongoose has a concept of connections, and it has a default connection. The default connection will be used for all models and schemas that you define.
The other connections need to be explicitly used.

Working with different databases

First, we need to define our collection schema:

export const LogSchema = new mongoose.Schema(
  {
    metadata: {
      type: mongoose.SchemaTypes.Mixed,
    },
  },
  {
    collection: 'Log',
    timestamps: true,
  },
);

LogSchema.name = 'Log';
LogSchema.databaseName = 'woovi-log';

const LogModel: Model<ILog> = mongoose.model('Log', LogSchema);

export default LogModel;

In the code above, we define the LogSchema, and the LogModel.
The LogModel is the LogSchema in the context of the default connection
To query Log collection from the default connection, we would do:

const logs = await LogModel.find();

To query logs from the woovi-logs database, we need a bit more work

const Log = getModelBySchema(LogSchema);

const logs = await Log.find();

We first need to create a model from the schema.
Let's see how getModelBySchema is implemented:

export const getConnection = (connectionName: string) => {
  const connection = mongoose.connections.find(
    (c) => c.name === connectionName,
  );

  if (!connection) {
    return mongoose;
  }

  return connection;
};

export const getModelBySchema = (schema) => {
  const connection = getConnection(schema.databaseName);

  return connection.model(schema.name, schema);
};

We first get the connection by name from mongoose.connections array, then we call connection.model to create a model from the schema.

To Conclude

As you scale you are going to need to break your database into more databases.
Creating the right abstraction can make this simple.

How many databases do you have in your product?

Woovi
Woovi is a Startup that enables shoppers to pay as they like. Woovi provides instant payment solutions for merchants to accept orders to make this possible.

If you want to work with us, we are hiring!

Image By rorozoa