4 Things I Learned from Mastering Mongoose.js

I first encountered MongoDB as part of the FreeCodeCamp curriculum, then as part of the MeteorJS stack, finally attending a talk at my bootcamp given by it's lead developer, Val Karpov (who, by the way, also coined the MEAN stack!).

Most resources, including my bootcamp, kind of treat Mongoose as a thin interface to MongoDB, which is a shame given how powerful it really is and how central your mastery of your database stack can be for an app's speed and scale. So it was of course exciting to hear that Val has finally written the definitive guide on Mongoose: Mastering Mongoose!

I'm going to jot down 4 things I learned. Note - I haven't used Mongoose in about 3 years, so this is mostly very introductory level.

1. Change Tracking for Minimal Updates

When you load a document from the database using a query and then modify it after, change tracking means Mongoose can determine the minimal update to send to MongoDB and avoid wasting network bandwidth.

// Mongoose loads the document from MongoDB and then _hydrates_ it
// into a full Mongoose document.
const doc = await MyModel.findOne();
doc.name; // "Jean Valjean"
doc.name = 'Monsieur Leblanc';
doc.modifiedPaths(); // ['name']
// `save()` only sends updated paths to MongoDB. Mongoose doesn't
// send `age`.
await doc.save();

This is very nice out of the box for performance and I don't have to do a thing!

2. Multiple Connections

Most apps only need one connection to MongoDB. However, Mongoose supports multiple connections:

const mongoose = require('mongoose');
const conn1 = mongoose.createConnection('mongodb://localhost:27017/db1',
 { useNewUrlParser: true });
const conn2 = mongoose.createConnection('mongodb://localhost:27017/db2',
 { useNewUrlParser: true });
// Will store data in the 'db1' database's 'tests' collection
const Model1 = conn1.model('Test', mongoose.Schema({ name: String }));
// Will store data in the 'db2' database's 'tests' collection
const Model2 = conn2.model('Test', mongoose.Schema({ name: String }));

This is helpful when:

• Your app needs to access data stored in multiple databases
• Your app has some slow operations and you don't want them to cause performance issues on fast queries. A MongoDB server can only execute a single operation on a given socket at a time, and the number of concurrent operations is limited by the poolSize of sockets. So to deal with slow connections, you can increase poolSize past the default of 5 - however, too many connections can hit OS-level performance issues and limits! Val uses poolSize = 10 for his production apps. Beyond that, you can't add more sockets. So it's better to just put slow operations on a separate connection.

3. Mongoose uses its own Middleware

In Mongoose, middleware lets you attach your own custom logic to built-in Mongoose functions. You can run pre and post any function:

• Document Middleware
  • validate()
  • save()
  • remove()
  • updateOne()
  • deleteOne()
• Model Middleware
  • insertMany()
• Aggregation Middleware
  • aggregate() - you can do fancy stuff here with modifying the aggregation pipeline
• Query Middleware
  • find()
  • count()
  • countDocuments()
  • deleteOne()
  • deleteMany()
  • distinct()
  • estimatedDocumentCount()
  • find()
  • findOne()
  • findOneAndDelete()
  • findOneAndRemove()
  • findOneAndReplace()
  • findOneAndUpdate()
  • remove()
  • replaceOne()
  • update()
  • updateMany()
  • updateOne()

This is a powerful pluggable system that reminds me of similar things in the npm scripts and Netlify Build system.

Mongoose uses its own system internally - Mongoose attaches a pre('save') middleware to all models that calls
validate(). That means save() triggers validate() middleware, which is why save() works the way it does.

4. Principle of Least Cardinality

With relational databases it is best to normalise data per schema. The third normal form is something like:

"Every non-key must provide a fact about the key, the whole key, and nothing but the key."

You can do this in Mongoose with populate(), but that goes against the grain of how NoSQL schemas should be setup. The common recommendation is to denormalize - a document should store all the properties you want to query by ("The Princple of Denormalization"), and, data that is referenced together, belongs together ("The Principle of Data Locality"). However, taken literally, this can lead to monster documents that can take forever to load.

The Principle of Least Cardinality states:

Store relationships in a way that minimizes the size of individual documents.

This helps make the correct tradeoff as far as Val is concerned. The other very good reason to do this is the fact that MongoDB limits documents to 16MB in size, of course - but primarily, the Principle of Least Cardinality is about conserving bandwidth when loading documents.

Conclusion

The book offers 4 sample apps built with Websockets, React, and Vue, and even ends with a great discussion about recommended app structure at the directory and app level! These polishing touches and the accessible and well thought through examples on every page make this a great reference point for anyone using Mongoose.js.