Manipulating arrays in MongoDB documents

Also published this on my blog

Introduction

In this article, we will study the update operators in MongoDB. Specifically, we'll expand on the array update operators.

I will use the mongo shell to show the examples. If you want to follow along in your terminal, you can find an installation guide here

Once everything is installed, you can launch a MongoDB instance with mongod. Then, in a new terminal window, type mongo (mongo.exe on windows) to start the shell.

We update documents in MongoDB by using the methods updateOne() or updateMany(). Both methods take two parameters. The first one is a filter that returns the document(s) which match this filter. The second specify the changes we want to apply to the returned document(s).

Every update operators starts with a $. Let's start with $push.

$push

This one might be quite easy to grasp, push is a popular method for lists across programming languages. It adds an element at the end of the array. If the array doesn't exist, it will create a new one and add the element.

Let's start with this document representing a user:

{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany"
}

We want to allow the user to add his favorite colors. At the moment, there is no field favoriteColors. Not to worry, $push will create it for us. Here is the syntax:

> db.user.updateOne(
    {"name": "Joe"}, 
    {$push: {"favoriteColors": "orange"}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "favoriteColors" : [
                "orange"
        ]
}

Note: the pretty() function can be used to nicely format the mongo shell output.
The second parameter of our updateOne function is an object that starts with our update operator $push. We give it as a value what we want to update in our document. We are basically saying:
Push the string orange at the end of the array favoriteColors
Let's push another color now:

> db.user.updateOne(
    {"name": "Joe"}, 
    {$push: {"favoriteColors": "blue"}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "favoriteColors" : [
                "orange",
                "blue"
        ]
}

Nice! Now, this is very simple form of push, one element at a time. MongoDB allows you to use several $-modifiers to perform more complex array operations. For example, if we want to push several items, we can use $each.

$each

Here is how you would add several items in one query:

> db.user.updateOne(
    {"name": "Joe"}, 
    {$push: {"siblings": { $each: ["Joey", "Sara", "Ben"]}}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "siblings" : [
                "Joey",
                "Sara",
                "Ben"
        ]
}

There you go, MongoDB mapped through the array specified in the $each modifier to add them one by one.

$slice

Let's add another modifier, $slice. Say we want our user to list 3 books. The array must not be bigger than 3. We can combine $push with $slice to achieve this:

> db.user.updateOne(
    {"name": "Joe"}, 
    {$push: 
        {"top3Books": 
            {$each: ["Book 1", "Book 2", "Book 3", "Book 4", "Book 5"], 
            $slice: -3 }}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "favoriteColors" : [
                "orange",
                "blue"
        ],
        "top3Books" : [
                "Book 3",
                "Book 4",
                "Book 5"
        ]
}

Our array will never be bigger than 3 elements. $slice keeps our three last entries because I gave it a value of -3. If I gave it a value of 3, we would have this document:

{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "favoriteColors" : [
                "orange",
                "blue"
        ],
        "top3Books" : [
                "Book 1",
                "Book 2",
                "Book 3"
        ]
}

You can control if new entries override the old ones or not. Of course, if we entered less than 3 items or 3 items, all of them would be in our array.
Next, let's say that we ask our user to rate the movies he saw. We want to keep only the ones with the higher ratings. To do this, we can combine our $push with an $each, then a $slice and finally a $sort. Like so:

> db.user.updateOne(
    {"name": "Joe"}, 
    {$push: {"top3Movies":{ $each: [
        {"name": "Star Wars VI", "rating": 5.3}, 
        {"name": "Her", "rating": 3.1},
        {"name": "John Wick 1", "rating": 7.8}, 
        {"name": "Jaws", "rating": 6.4}, 
        {"name": "Saw", "rating": 2.3}, 
        {"name": "The Lord of the rings", "rating": 8.4}, 
        {"name": "The Hobbit", "rating": 4.3}], 
        $slice: -3, 
        $sort: {"rating": 1 }}}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "top3Movies" : [
                {
                        "name" : "Jaws",
                        "rating" : 6.4
                },
                {
                        "name" : "John Wick 1",
                        "rating" : 7.8
                },
                {
                        "name" : "The Lord of the rings",
                        "rating" : 8.4
                }
        ]
}

Let's break this down a bit. As always, we started with the $push operator. We inform MongoDB that the field name is "top3Movies". We specify to our $each operator an array of documents we want to push to our field. Each document has a name and a rating field. Next, we use $slice: -3 to indicates that we wan't to keep only 3 documents in this array. Finally, we use $sort on the rating field to keep the 3 movies with the higher ratings.
Note: If you gave -1 as a value for the sort field, we would have the 3 lowest ratings in our array.

Array as set

Maybe you want to treat your array as a set. A set is a list where there are no duplicates. There are two ways you could do this, by using $ne or $addToSet.

$ne

$ne is used in the first parameter, the filter query, like this:

> db.user.updateOne(
    {"favoriteColors": {$ne : "orange"}},
    {$push: {"favoriteColors": "orange"}} )
{ "acknowledged" : true, "matchedCount" : 0, "modifiedCount" : 0 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "favoriteColors" : [
                "orange",
                "blue"
        ]
}

We are saying to MongoDB, check if favoriteColors contains the value "orange". If not, push "orange" else do nothing. Because we already added orange to this document earlier on, the document didn't change ( modifiedCount: 0 ).

$addToSet

You may want to use $addToSet when $ne can't be used, or doesn't describe what you want. My user has several email addresses, but I wan't to make sure that they are all unique in my document.

> db.user.updateOne(
    {"name":"Joe"}, 
    {$addToSet: { "emails" : {$each: ["joe@hotmail.com", "joe@yahoo.com", "joe@gmail.com"]}}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "emails" : [
                "joe@hotmail.com",
                "joe@yahoo.com",
                "joe@gmail.com"
        ]
}

The $addToSet function like a $push but checks if the value is already inside the array. Let's try to add an already existing email address:

> db.user.updateOne({"name": "Joe"}, {$addToSet: {"emails" : "joe@hotmail.com"}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 0 }
> db.user.find().pretty()
{
        "_id" : ObjectId("595fee69e331f76e40447a33"),
        "name" : "Joe",
        "age" : 35,
        "country" : "Germany",
        "emails" : [
                "joe@hotmail.com",
                "joe@yahoo.com",
                "joe@gmail.com"
        ]
}

Yeah! No duplicates ( modifiedCount: 0 ). As we saw, you can use $addToSet in conjunction with $each. You could also use $slice or $sort with $addToSet.

 

That's it for the first part. In the next article, we'll explore removing elements from an array and modify the position of elements.