Express.js on Netlify

Netlify started off as a service for hosting static websites. Not only was it an amazing service from the start, managing the hosting infrastructure so you can focus on putting your website together, but they have since added support for those looking to go beyond a static site. AWS (Amazon Web Services) Lambda functions, their most recent add-on, means developers now have the freedom to run arbitrary back-end code to support their website without having to leave Netlify.

However, if you come from an Express.js background, you may not be interested in working with or have the time to learn a new technology stack. In this post, I will show how you can build an Express.js application on top of Lambda functions. If you are using another web framework (e.g. Koa.js), stick around! The contents of this post should also apply to any web framework built upon Node.js’s http module.

Although I will be referring to only AWS Lambda functions in this post, I really mean the API Gateway API and Lambda integration, which Netlify functions has abstracted away for us.

Quick Introduction to Lambda Functions

Imagine a function that updates a database in a web application that is running on a server:

function updateDatabase(data) {
  ... // update the database
  return newValue;
}

and imagine that it was tied to a route:

app.post('/updatestate', (res, req) => {
  const newValue = updateDatabase(res.body);
  req.json(newValue);
});

How many servers do you need to serve the /updatestate endpoint? Do you know when to add servers during high-load periods and when to scale down to save money?

The idea behind Lambda functions is that you simply pay for what you need and do not have to worry about responding to changes in load to your application. If nothing is happening, you do not pay at all. If the number of requests spike, AWS automatically scales for you in the background to match that load. All you have to do is provide the code you want Lambda to execute upon each request. In our example that would be the equivalent of updateDatabase(), which would look something like this:

'use strict';
function updateDatabase(data) {
  ... // update the database
  return newValue;
}

exports.handler = function(event, context, callback) {
  if(event.httpMethod === 'POST' && event.path === '/my/path') {
    const requestBody = JSON.parse(event.body);
    const newValue = updateDatabase(requestBody);
    callback(null, {
      statusCode: 200,
      body: newValue
    });
  } else {
    callback(null, {
      statusCode: 400,
      body: {}
    });
  }
}

Lambda will call the handler function for each request. The event object is Lambda’s way of representing the incoming request, and the callback(err, data)function is how you tell Lambda what response to send back.

Express.js via serverless-http

Express.js is a web framework that is built on the http module, simplifying route management as described in Understanding Express.js by Evan Hahn. If you look at the following example from that article, you will notice that you have to manage routing when using http, similar to when using Lambda functions:

var http = require("http");

http.createServer(function(req, res) {
  // Homepage
  if (req.url === "/") {
    res.writeHead(200, { "Content-Type": "text/html" });
    res.end("Welcome to the homepage!");
  }

  // About page
  else if (req.url === "/about") {
    res.writeHead(200, { "Content-Type": "text/html" });
    res.end("Welcome to the about page!");
  }

  // 404'd!
  else {
    res.writeHead(404, { "Content-Type": "text/plain" });
    res.end("404 error! File not found.");
  }
}).listen(1337, "localhost");

Going back to our Lambda function, you will notice that the event object andcallback function are very similar to Express.js’s req and res object, in that they facilitate getting information about the request and returning a response to the client. In fact, it is similar enough that a library exists to translate between the two:serverless-http.

With serverless-http, you can do something like this:

'use strict';
const express = require('express');
const serverless = require('serverless-http');
const app = express();
const bodyParser = require('body-parser');

function updateDatabase(data) {
  ... // update the database
  return newValue;
}

app.use(bodyParser);
app.post('/updatestate', (res, req) => {
  const newValue = updateDatabase(res.body);
  req.json(newValue);
});

module.exports.handler = serverless(app);

The key is that last bit, where serverless-http acts as a decorator and translates Lambda’s event into something Express.js can use, and req back to a proper call to Lambda’s callback(err, data).

Benefits

Using serverless-http and Express.js on top of Lambda functions simplifies the routing decision code, similiar to why you would use Express.js over the httpmodule. It also allows you to use your favourite Express.js patterns and middleware for your Netlify websites without having to leave Netlify.

Limitations

There are several limitations to this approach.

Resource Limits

Netlify’s Lambda functions have a 128 MB memory and 10 seconds execution limit. In terms of processing power (CPU), AWS does not disclose that value, but it is proportional to how much memory is allocated to the Lambda function. There is also 512 MB of temporary disk space available across all memory configurations.

These limits mean you will not be able to have any large or long-running back-end requests. If you bypass those limits, the Lambda function stops executing immediately with an error, which translates to a 500 HTTP status code. You can see the full set of limits on AWS’s developer guide.

Interestingly enough, 1024 MB Lambda functions are more price-effective than 128 MB Lambda functions and they run in a shorter time. If you are a heavy user of Lambda functions, you may want to contact sales as per their documentation to request 1024 MB functions.

Statelessness

AWS makes no guarantees that any instance of a Lambda function will be reused for multiple requests, and can create a new instance at any time. This means you can not expect any in-memory state or file system content to persist across requests. For more details, I recommend looking at the developer guide.

Static Files

Netlify requires that each Lambda function’s code to be self-contained in one file. This means you are unable to serve any static files from Express.js, and that any view templates will have to be embedded in JavaScript files.

Database

Behind most web applications is a database. Netlify does not provide a database service (yet), so if you need a persistent data store, such as for session management, you will have to bring your own at this time.

Keep in mind that there are no guarantees that subsequent calls will reuse an instance of your Lambda function, so do not rely on in-memory caches (e.g. do not use memorystore to manage your sessions).

Conclusion

In this post, I have showed how you can use Express.js with Lambda functions and outlined the limitations with this approach. If you are interested in exploring this topic further, I have a very simple deploy-ready sample at neverendingqs/netlify-express that you can get started with.

Let me know what you think by leaving a comment below!