While working on a MERN stack project, I came across a situation where I wanted to populate a field but also populate a field inside that populated field (I know it's confusing. Bear with me :p ). So, I solved it and decided to share it with you all. Nice idea, isn't it ? Let's get started then !
I assume you know the basics of mongoose, mongodb and nodejs. In this post, I will cover populate. What it is, How it works, and how to use it to populate documents in mongodb.
What is Population ??
Population is way of automatically replacing a path in document with actual documents from other collections. E.g. Replace the user id in a document with the data of that user. Mongoose has an awesome method populate to help us. We define refs in ours schema and mongoose uses those refs to look for documents in other collection.
Some points about populate:
- If no document is found to populate, then field will be
null. - In case of array of documents, if documents are not found, it will be an empty array.
- You can chain populate method for populating multiple fields.
- If two populate methods, populate same field, second populate overrides the first one.
First things first. We need an example to work with !!
We are going to create 3 collections with 3 schemas:
- User
- Blog
- Comment
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const UserSchema = new Schema({
name: String,
email: String,
blogs: [{
type: mongoose.Schema.Types.ObjectId,
ref: "Blog"
}]
});
const BlogSchema = new Schema({
title: String,
user: {
type: mongoose.Schema.Types.ObjectId,
ref: "User"
},
body: String,
comments: [{
type: mongoose.Schema.Types.ObjectId,
ref: "Comment"
}]
})
const CommentSchema = new Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: "User"
},
blog: {
type: mongoose.Schema.Types.ObjectId,
ref: "Blog"
},
body: String
})
const User = mongoose.model("Author", UserSchema);
const Blog = mongoose.model("Blog", BlogSchema);
const Comment = mongoose.model("Comment", CommentSchema);
module.exports = {User, Blog, Comment}
Minimal schemas with references to other schemas that will help us use populate method.
How Populate Works
Now let's see how populate works. I won't be writing the whole code. Only the important parts.
Suppose you want a user by id with its blogs. If you do it without populate, you will get the user document with his blog ids array. But we want blog documents instead of ids !!
So let's see how to do it.
// in your node js file
const User = require("path/to/userSchema");
User
.findOne({_id: userId })
.populate("blogs") // key to populate
.then(user => {
res.json(user);
});
/*
OUTPUT:
{
_id: userid, // obviously it will be id generated by mongo
name: "john doe",
email: "john@email.com",
blogs: [
{
_id: blogid,
title: "how to do nothing",
body: "Interesting matter in 11111the blog...",
comments: [commentId_1, commentId_2]
}
]
}
*/
Easy right? Populate is awesome for joining documents like that. You will get user with all blog documents in blogs array.
But if you see the output you will notice comments array is still full of comment ids instead of documents from comments. How do we populate them ??? Keep reading to know...
Nested Populate in a document !
Let's see how to do nested populate in a query and populate comments in user blogs.
User
.findOne({_id: userId })
.populate({
path: "blogs", // populate blogs
populate: {
path: "comments" // in blogs, populate comments
}
})
.then(user => {
res.json(user);
});
/*
OUTPUT:
{
_id: userid, // obviously it will be id generated by mongo
name: "john doe",
email: "john@email.com",
blogs: [
{
_id: blogid,
title: "how to do nothing",
body: "Interesting matter in the blog...",
comments: [
{
user: userId,
blog: blogId,
body: "your blog is awesome !"
}
]
}
]
}
*/
So that was it. If you want to select specific fields while populating, you can use select key to specify fields inside an object.
// simple populate
User
.findOne({_id: userId })
.populate("blogs", { name: 1 }) // get name only
// nested populate
User
.findOne({_id: userId})
.populate({
path: "blogs",
populate: {
path: "comments",
select: { body: 1 }
}
})
EXTRA : Populate After Save !?
Sometimes (rarely), you may want to populate a document after saving it to mongodb. Example, you create a new comment and save it, but when you send it with response you want to add user info in it instead of just user id.
const Comment = require("/path/to/commentSchema");
let newComment = new Comment({
user: userId,
blog: blogId,
body: "this is a new comment"
});
newComment.save().then(result => {
Comment
.populate(newComment, { path: "user" })
.then(comment => {
res.json({
message: "Comment added",
comment
});
})
})
/*
OUTPUT: Comment
{
user: {
_id: userid,
name: "john doe",
email: "johndoe@email.com",
blogs: [blogId_1, blogId_2
},
blog: blogId,
body: "your blog is awesome !"
}
*/
Any Suggestions are much appreciated.
Hope you find it useful and learned something new from it.
Happy Coding :)
Discussion (19)
From the outside, this seems like basically creating
JOINfunctionality from an RDBMS. That's not a bad thing! It turns out that sometimes an RDBMS needs document-style records, and sometimes document DBs need relations :) Just wanted to make the comparison in case it helps anyone else (and to make sure I understand it correctly).Correct !!...And Nice comparision. :)
Awesome . I knew this In django , Finally Got one for Node 🤘
does populate in mongoose issue a separate DB call for fetching the referenced documents or does it translate to a $lookup and issue a single query?
No, it doesn't translate to a $lookup and Yes, it makes another query to db to get the required data.
Good question !
I see. yeah clientside joins are a difficult problem to solve. I've been trying to do that with my c# mongodb library but still haven't found a proper/ user friendly way. was expecting to steal it from mongoose if they've managed it 😜
hahaha...i think you have to use $lookup and $aggregation :p
Glad you found t, It's very helpfull
Thank you !! :)
i don't know why populate is not working for me i get a empty array
don't forget to add ids in that array
yes that's why i was getting an empty array, thank you
:) good luck !
Cool tip didn’t know this one.
Glad you found it useful :)
Absolute delight to read.
thanks !!
I wonder why the populate method totally refused to work in my Express.js project. Tried everything as stated in the Mongoose Populate() documentation, yet nothing works.
really ?!.. what error are you getting ?