Understanding MongoDB Aggregation Pipeline

In most of my REST API I have built, MongoDB has been my go to NoSQL database. MongoDB is arguably the most popular NoSQL database in the world.
In most of my use cases, I typically use the find() command for a wide range of queries. However, I realized the need to learn more about MongoDB aggregation in my most recent projects where I needed to write complex queries.

In this article, I'll explain the key stages in MongoDB's aggregation pipeline and how to apply them to the pipeline

What is Aggregation Pipeline?

Aggregation is the process of going through various stages with a huge collection of documents to process them. The states is what is termed as pipeline.
The aggregation pipeline in MongoDB has several stages, each of which modifies the document. Or, to put it another way, the aggregation pipeline is a multi-stage pipeline, so in each state, the documents taken as input and produce the resultant set of documents now in the next step (if available), the resultant documents taken as input and produce output, and so on until the last stage.

Below is an image illustration of how MongoDB Aggregation pipeline works.

Stages in Aggregation pipelines

There are several stage in MongoDB Aggregation Pipelines. In this section, I will be explaining a few of the stages that are commonly used.

• $match - Filters the documents to pass only the documents that match the specified condition(s) to the next pipeline stage. This can reduce the amount of documents that are given as input to the next stage. Syntax

db.users.aggregate(
    [ { $match : { email : "example@gmail.com" } } ]
);

• $group - It is use to group documents from collections based on a 'group key' Syntax

db.sales.aggregate( [
  {
    $group: {
       _id: null,
       count: { $count: { } }
    }
  }
] )

• $sort - This stage is used to rearrange the resulting document (Ascending or Descending) Syntax

db.contestants.aggregate(
   [
     { $sort : { votes : 1 } }
   ]
)

• $project - It is used to select some specific fields from the resulting collection to be sent to the client. Syntax

db.events.aggregate(
   [
        $project: {
           _id: 0,
           name: 1,
           description: 1,
           banner: 1,
           opening_date: 1,
           closing_date: 1,
        },
   ]
)

Note: 1 assigned to a field means inclusive. This will include that field in the output document. 0 is the vice versa. It will exclude the field from the document.
It is important to note that non-zero values also considered as inclusive.

• $lookup - Performs a left outer join to a collection in the same database to filter in documents from the "joined" collection for processing. Syntax

db.orders.aggregate( [
   {
     $lookup:
       {
         from: "inventory",
         localField: "item",
         foreignField: "sku",
         as: "inventory_docs"
       }
  }
] )

• $unwind - Deconstructs an array field from the input documents to output a document for each element. Each output document is the input document with the value of the array field replaced by the element. Syntax

db.events.aggregate( [ { $unwind : "$organizer" } ] )

Each of the Stages has their prototype and how to implement them.
You can explore them more using
Documatic VScode Extension
This extension brings Documatic to VSCode: quickly search your large codebases using simple queries - what does it do? what dependencies does it have? And more.

Documatic search uses AI to link relations between your query and snippets of code, so you don't have to know the exact keywords you're looking for!

Sample usecase.

In this use case, There are two models or collections, Events and their Organizers.
Each event document has it's organizer _id as a foreign key.
I used aggregation pipeline to fetch events together with their organizer data instead of just the organizer id.

This snippet makes use of a combination of the stages discussed about.

       const events = await Event.aggregate([
            {
                $lookup: {
                    from: "organizers",
                    localField: "organizer",
                    foreignField: "_id",
                    as: "organizer",
                },
            },
            {
                $unwind: "$organizer",
            },
            {
                $project: {
                    name: 1,
                    description: 1,
                    banner: 1,
                    vote_price: 1,
                    opening_date: 1,
                    closing_date: 1,
                    organizer: {
                        name: "$organizer.name",
                        email: "$organizer.email",
                        phone: "$organizer.phone",
                        company: "$organizer.company",
                        address: "$organizer.address",
                    },
                },
            },
        ]);


        res.status(200).json({
            success: true,
            data: events,
            message: "Events fetched successfully",
        });

OUTPUT

Conclusion

I have discussed with you some of the stages of MongoDB Aggregation Pipeline with examples of how to use them.

As you use MongoDB for more complex projects, Aggregation will become an inevitable concept to perform your advanced querying which forms integral part of your API.

Happy Hacking!

Bentil here🚀
Have you used Aggregation Pipelines in your project before? What was the experience? I will be glad to hear from you in the comment section. This will help other who are yet to use Aggregation in their projects.

If you find this content helpful, Please Like, comment and share.