Building a RESTful API with Express and MongoDB

Today we're continuing with our exploration of Express. In the last tutorial we created a basic website with Express.js. Now we're going to build an API using express and mongoDB. The data we'll be working with are books. By the end of the tutorial we'll have a REST API which allows us to read, write, edit and delete content from a mongo database. In the process we'll explore all the major Verbs associated with APIs.

The term Representational State Transfer (REST) was coined by Roy Fielding back in 2000. It lays out a set of rules which we'll need to follow when creating APIs. Having these rules in place means other developers which will eventually use the API know exactly what to expect.

Lets get started

First thing we need to do is set up the project by initialising with npm and installing the packages we are going to use.

npm init
npm install --save express mongoose

If you are new at express you should go through the tutorial I mentioned above, then come back here.

Just to quickly get the project started we're going to use the same code as in the previous tutorial. Create a server.js file and add the following

import express from 'express';
const app = express();
const port = process.env.PORT || 5656;
// routes go here
app.listen(port, () => {
    console.log(`http://localhost:${port}`)
})

That's exactly what we worked with in the previous tutorial. That would start a server at port 5656 if an alternative is not specified.

The way APIs work is that they control what is made available to the clients that make requests.

In our case, we are creating a book API, which means we are giving other developers access to the book information we have in a database.

Display books

Amongst other things we are going to make our data available when ever a GET request is made to our API. The data which we are going to respond with will need to be in JSON format.

app.get('/api/books', (req, res) => {
    res.json([
            {
                id: 1,
                title: "Alice's Adventures in Wonderland",
                author: "Charles Lutwidge Dodgson"
            },
            {
                id: 2,
                title: Einstein's Dreams",
                author: "Alan Lightman"
            }
        ])
})
app.get('/api/books/2', (req,res)=>{
    res.json(
            {
                id: 2,
                title: Einstein's Dreams",
                author: "Alan Lightman"
            }
        )
})

We'll shortly get that information from a database but that's for illustration purposes. /api/books returns all books and /api/books/2 returns a book with the ID of two.

Refactoring Code

Before we go any further, let's refactor the code to make it more manageable in the future.

If we continue adding all the API verbs in this fashion the server.js file will become very messy and the code hard to read. Express gives us the ability to write our routes in a separate file and include them in our project.

Let's create a file at Routes/bookRouter.js and move the two routes from server.js like so:

import express from 'express';
const bookRouter = express.Router();
bookRouter
    .get('/', (req,res) => {
        res.json(...)
    })
    .get('/2', (req,res) => {
        res.json(...)
    })
export default bookRouter;

Note how in server.js we attached the two GET routes to app, which referenced express(). Express gives us Router() precisely to enable us to organise our routes. So in this case, all our book routes will be appended to bookRouter.

Finally, we need to import that file to server.js and use those routes in our express application.

import bookRouter from './Routes/bookRouter';
...
app.use('/api/Books', bookRouter);

From here on, all the routes are going to be added in Routes/bookRouter.js.

Working with mongoDB

Above we explored the basic structure that our API will take, but realistically the books have to be stored in a database for this API to be useful.

If you never used mongoDB before then most likely you don't have it installed in your computer. I highly recommend you do so but that's beyond the scope of this tutorial. As a result we're going to go with an alternative provider: mlab.com. When we register with mlab, we get a free mongo database, with plenty of free space for what we need.

It will ask you to select a provider (pick any of the three), then select "sandbox" plan (which is free), then hit continue. Then you are asked to select a region. I read that in some regions the free plan doesn't exist! So just keep the preselected region.

Finally add a name to your database. Then submit your order. Note, if you get an error requesting you to pick a different name, just edit the database name.

Click on the database name, then select 'Users'. Here you need to add a user. This user is the one that will have access to the database.

Connecting to the database

We are going to use the mongoose package to connect and manipulate the database. We've already installed mongoose at the beginning.

These are the steps we need to take in order to gain access to the database using mongoose:

1. Connect to the database
2. Build a model
3. Perform operations

In the spirit of modularity, we'll connect to the database on the server.js

import mongoose from 'mongoose';
const db = mongoose.connect('mongodb://<dbuser>:<dbpassword>@ds125068.mlab.com:25068/api-test2');

Then build the model in its separate file at models/bookModel.js

import mongoose from 'mongoose';
const Schema = mongoose.Schema;
const bookModel = new Schema({
    title: { type: String   },
    author: { type: String }
})
export default mongoose.model('books', bookModel)

MongoDB works with collections (MySql has tables). The mongoose model requires two arguments: a collection name, and a schema. In the schema we get to specify the fields we want to have in our database collection. As we'll see, the above setup ensures that regardless of what data is passed to mongoose, it will only accept a title and author and seemingly ignore other properties.

GET: Getting books from database

Finally, from this point on, every request made to the API will commmunicate with the database. Lets change the two GET routes we've previously created at Routes/bookRouter.js to reflect that.

import express from 'express';
import Book from '../models/bookModel';
const bookRouter = express.Router();
bookRouter.route('/')
    .get((req, res) => {
        Book.find({}, (err, books) => {
            res.json(books)
        })  
    })
bookRouter.route('/:bookId')
    .get((req, res) => {
        Book.findById(req.params.bookId, (err, book) => {
            res.json(book)
        })  
    })

First we import the schema. That schema then gives us the entire mongo methods associated with collections (which you can find in mongo's docs).

Since we'll be chaining all the request methods together, we are defining the route endpoints using the route method.

In the first router configuration we get all the books. find() takes two arguments, a query and a callback function. The query is an object used to filter the data. Note that we passed a blank object, hence, we get all the books.

The second get route Is a lot more interesting. /:bookId can be described as a placeholder I guess. If we navigate to /api/books/SomeOtherPage, then SomeOtherPage can be referenced by /:bookId - this is a feature of express. req.params.bookId then would equal SomeOtherPage. In reality we are expecting the book ID to match the ID auto generated in the database.

POST: Adding content to the database

POST is used to add new content to the database. In our case, we'll use it to add new books. Still in Routes/bookRouter.js we are able to chain post() after get()

...
bookRouter.route('/')
    .get((req, res) => { ...})
    .post((req, res) => {
        let book = new Book({title: 'The Bull', author: 'Saki'});
        book.save();
        res.status(201).send(book) 
    })

mongoose makes working with the database very easy. We create a new book, save it to the database and pass it back to the client.

As you've noticed, this is not very useful. Ideally, the title and author should be retrieved from the request. That information should be provided by the API users.

In order for us to be able to read the data that comes with the request, we need to use an express middleware which enables the parsing of incoming data. The parsing middleware is called body-parser, it is a package which we need to install through NPM.

npm install --save body-parser

Now we are able to use this middleware in our express code. Lets do so in ./server.js.

import bodyParser from 'body-parser';
...
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

First line, we import the package. In second and third lines, bodyParser looks at the incoming data and parses it depending on whether it's JSON or data coming from a form.

Back to our POST route, body-parser has attached the incoming data to the request object.

...
bookRouter.route('/')
    .get((req, res) => { ...})
    .post((req,res) => {
        let book = new Book(req.body); // edited line
        book.save()
        res.status(201).send(book)
    })

That's it, as long as a title and an author comes through, it is added to the database, and the same book is passed back.

Remember, mongoose schema knows what it needs from the provided body content. If req.body has more data than mongoose schema needs, that data is ignored. If, say, the author doesn't exist in the req.body then an object with only the title would be added to the database, author wouldn't exist at all, mongo is very flexible that way.

PUT: Editing books in the database

We use PUT to edit a specific entry. In the case of our books PUT is used to edit one book. Hence, we chane put() to the /:bookId route

bookRouter.route('/:bookId')
    .get(...)
    .put((req,res) => {
        Book.findById(req.params.bookId, (err, book) => {
            book.title = req.body.title;
            book.author = req.body.author;
            book.save()
            res.json(book)
        }) 
    })

As we'll see, the way we interact with the database whilst performing different operations is very similar to one another. Above we find a book and we change the the properties of the book object stored in the database with those that are passed along with the request.

PATCH: Editing book properties

PATCH will allow users to edit specific properties of a book object. This is still attached to bookRouter.route('/:bookId'). We pull the particular book from the database and modify all the properties that match the incoming information

bookRouter.route('/:bookId')
    .get(...)
    .put(...)
    .patch((req,res)=>{
        Book.findById(req.params.bookId, (err, book) => {
            if(req.body._id){
                delete req.body._id;
            }
            for( let b in req.body ){
                book[b] = req.body[b];
            }
            book.save();
            res.json(book);
        })
    })

Users of our API make a PATCH request to /api/books/5a76f7373ec6426aaeb91146, with it they pass the information they want to change - they might want to change the author's name for example.

If they pass an _id as one of the properties they want to edit we ignore that request as the IDs are unique identifiers used to organise data, amongts other things, and shouldn't be changed.

Then the for loop loops through the remaining properties from the incoming object and updates the properties found in the database with those coming through the request.

DELETE: Removing book

Finally we want users to be able to delete an entire book

bookRouter.route('/:bookId')
    .get(...)
    .put(...)
    .patch(...)
    .delete((req,res)=>{
        Book.findById(req.params.bookId, (err, book) => {
            book.remove(err => {
                if(err){
                    res.status(500).send(err)
                }
                else{
                    res.status(204).send('removed')
                }
            })
        })
    })//delete

As usual, we find the particular book by its ID, and then mongoose gives us the ability to remove it simply by running the remove() method on the found book.

Refactoring: using custom middleware

Whilst writing the code for get, put, patch and delete methods on /api/books/:bookId I'm sure you've noticed that we repeat the same code which interacts with the database. I believe seeing it repeated would clarify the the fact that the code for each method retrieves data from the database in the same manner.

We've already used a middleware in our code - body-parser. The way middlewares work is they run before our code!

bookRouter.use('/:bookId', (req, res, next)=>{
    console.log("I run first")
    next()
})
bookRouter.route('/:bookId')
    .get((req,res)=>{
        Book.findById(req.params.bookId, (err, books) => {
            res.json(books)
        })
    })

When a get request is made to /api/books/:bookId the message is logged (in the terminal) then our code which responds to the get request runs.

Note, if we don't include the next() method, the get request code never executes!

Let's take advantage of that middleware to make something useful.

bookRouter.use('/:bookId', (req, res, next)=>{
    Book.findById( req.params.bookId, (err,book)=>{
        if(err)
            res.status(500).send(err)
        else {
            req.book = book;
            next()
        }
    })

})

We used the middleware to retrieve the required book from the database, if successful, the book object is attached to the request object.

With that in place, let's modify all the verbs associated with /:bookId route.

bookRouter.route('/:bookId')
    .get((req, res) => {
        res.json(req.book)
    }) // end get Books/:bookId 
    .put((req,res) => {
        req.book.title = req.body.title;
        req.book.author = req.body.author;
        req.book.save()
        res.json(req.book)
    })
    .patch((req,res)=>{
        if(req.body._id){
            delete req.body._id;
        }
        for( let p in req.body ){
            req.book[p] = req.body[p]
        }
        req.book.save()
        res.json(req.book)
    })//patch
    .delete((req,res)=>{
        req.book.remove(err => {
            if(err){
                res.status(500).send(err)
            }
            else{
                res.status(204).send('removed')
            }
        })
    })//delete

The difference being that the database interaction is writen once in the middleware but still runs on every request.

That's it. The REST API is complete.

Postman

To test if all the routes work, you can use Postman, a chrome application

Heres how a post request can be made

At the top we select the method/verb - above we selected POST. Then enter the API endpoint. Then simply select Headers and add application/json as the Content-Type and in the body (with raw option selected) you can pass the JSON content you wish to pass to the API. In the above screenshot we are passing the required title and author.

Thanks for reading

The code can be downloaded from github