Using array map, filter and reduce in MongoDB aggregation pipeline

If you work with javascript, the chances of you are using array methods like map, filter and reducer today are really great.

All simplicity offered by javascript higher-order functions makes our code more readable and concise, mainly when we work with array data transformations.

Let's remember these methods:

const numbers = [2, 8, 15];

const greaterThanFive = (num) => num > 5;
const multiplyBy2 = (num) => num * 2;
const sum = (acc, num) => acc + num;

const filtered = numbers.filter(greaterThanFive);
const mapped = numbers.map(multiplyBy2);
const reduced = numbers.reduce(sum);

console.log(filtered); // [8, 15]
console.log(mapped); // [4, 16, 30]
console.log(reduced); // 25

That's really amazing!

However, in databases scenario, querying data with this simplicity is usually unusual, unless that database is MongoDB.

Because MongoDB is a NoSQL database with JSON based model, some javascript array methods have similar expression operators
in MongoDB Aggregation Pipeline.

About its JSON nature, official website cites:

MongoDB’s document data model naturally supports JSON and its expressive query language is simple for developers to learn and use.

And that makes all difference folks...

Let's get numbers array data used in the javascript example to create a new document in a generic collection. For improve the understanding, I will use MongoDB Playground to test our queries:

[
  {
    "numbers": [
      2,
      8,
      15
    ]
  },
]

Mongo Playground

Good! Our collection is ready to receive queries now :)

$filter

Starting, let's use $filter aggregation pipeline operator.

Query

db.collection.aggregate([
  {
    $project: {
      _id: 0,
      filtered: {
        $filter: {
          input: "$numbers",
          as: "num",
          cond: {
            $gt: [
              "$$num",
              5
            ]
          }
        }
      }
    }
  }
])

• Start using aggregate method to submit the query. That method enables aggregation framework;
• Pipeline starts using $project aggregation pipeline stage. The specified fields inside it can be existing fields from the input documents or newly computed fields. In our case, filtered field will be created and added to response;
• The computed value for filtered field will be given by $filter aggregation pipeline operator;
• Inside filter operator, set input to $numbers. That's our array to be iterated;
• Set as to num to get each array value to test in filter condition. You could use any name here, just like you did in javascript filter method;
• Then, set the filter condition in cond using $gt expression to return a boolean if current array value $$num is greater than 5;

Response

[
  {
    "filtered": [
      8,
      15
    ]
  }
]

Mongo Playground

$map

$map operator is pretty similar to $filter, however while $filter needs a condition, $map you must set in to output a new array value according to some rule.

Query

db.collection.aggregate([
  {
    $project: {
      _id: 0,
      mapped: {
        $map: {
          input: "$numbers",
          as: "num",
          in: {
            $multiply: [
              "$$num",
              2
            ]
          }
        }
      }
    }
  }
])

In case, using $multiply expression to return all array values multiplied by 2.

Response

[
  {
    "mapped": [
      4,
      16,
      30
    ]
  }
]

Mongo Playground

$reduce

$reduce operator applies an expression to each element in an array and combines them into a single value.

Query

db.collection.aggregate([
  {
    $project: {
      _id: 0,
      reduced: {
        $reduce: {
          input: "$numbers",
          initialValue: 0,
          in: {
            $sum: [
              "$$value",
              "$$this"
            ]
          }
        }
      }
    }
  }
])

• Again, set $numbers array as input to iterate;
• The initial cumulative value set before in is applied to the first element of the input array, initialValue is set to 0;
• Finally, in expression give us two special variables: $$value is the variable that represents the cumulative value of the expression (acc in javascript example) and $$this is the variable that refers to the element being processed (num in javascript example). In case, using $sum expression to return the new accumulated value.

Response

[
  {
    "reduced": 25
  }
]

Mongo Playground

All in one

In previous examples, we worked with each operator in a separated query, however we could do a single query requesting all operators at once.

Query

db.collection.aggregate([
  {
    $project: {
      _id: 0,
      filtered: {
        $filter: {
          input: "$numbers",
          as: "num",
          cond: {
            $gte: [
              "$$num",
              5
            ]
          },

        }
      },
      mapped: {
        $map: {
          input: "$numbers",
          as: "num",
          in: {
            $multiply: [
              "$$num",
              2
            ]
          }
        }
      },
      reduced: {
        $reduce: {
          input: "$numbers",
          initialValue: 0,
          in: {
            $sum: [
              "$$value",
              "$$this"
            ]
          }
        }
      }
    }
  }
])

Response

[
  {
    "filtered": [
      8,
      15
    ],
    "mapped": [
      4,
      16,
      30
    ],
    "reduced": 25
  }
]

Mongo Playground

Going further, if you add more documents to collection, this same query computes data for each of them. Let's query a collection with 3 documents now:

Collection

[
  {
    "numbers": [
      2,
      8,
      15
    ]
  },
  {
    "numbers": [
      4,
      8,
      9,
      13
    ]
  },
  {
    "numbers": [
      1,
      3,
      7
    ]
  }
]

Response

[
  {
    "filtered": [
      8,
      15
    ],
    "mapped": [
      4,
      16,
      30
    ],
    "reduced": 25
  },
  {
    "filtered": [
      8,
      9,
      13
    ],
    "mapped": [
      8,
      16,
      18,
      26
    ],
    "reduced": 34
  },
  {
    "filtered": [
      7
    ],
    "mapped": [
      2,
      6,
      14
    ],
    "reduced": 11
  }
]

Mongo Playground

Conclusion

MongoDB for javascript developers is intuitive by nature! Aggregation framework does the hard work directly in the database server using many of features already known by us and data can be delivered ready-to-use, which normally decreases the workload for the application server.

See also the complete Array Expression Operators list in MongoDB official website.