Migrating a classic Express.js to Serverless Framework

Problem

Classic Express.js applications are great for building backends. However, their deployment can be a bit tricky. There are several solutions on the market for making deployment "easier" like Heroku, AWS Elastic Beanstalk, Qovery, and Vercel. However, "easier" means special configurations or higher service costs.

In our case, we were trying to deploy an Angular frontend served through Cloudfront, and needed a separately deployed backend to manage an OAuth flow. We needed an easy to deploy solution that supported HTTPS, and could be automated via CI.

Serverless Framework

The Serverless Framework is a framework for building and deploying applications onto AWS Lambda, and it allowed us to easily migrate and deploy our Express.js server at a low cost with long-term maintainability. This was so simple that it only took us an hour to migrate our existing API, and get it deployed so we could start using it in our production environment.

Serverless Init Script

To start this process, we used the Serverless CLI to initialize a new Serverless Express.js project. This is an example of the settings we chose for our application:

$ serverless
What do you want to make? AWS - Node.js - Express API
What do you want to call this project? example-serverless-express

Downloading "aws-node-express-api" template...
Installing dependencies with "npm" in "example-serverless-express" folder
Project successfully created in example-serverless-express folder

What org do you want to add this service to? [Skip]
Do you want to deploy your project? No

Your project is ready for deployment and available in ./example-serverless-express
Run **serverless deploy** in the project directory
    Deploy your newly created service
Run **serverless info** in the project directory after deployment
    View your endpoints and services
Run **serverless invoke** and **serverless logs** in the project directory after deployment
    Invoke your functions directly and view the logs
Run **serverless** in the project directory
    Add metrics, alerts, and a log explorer, by enabling the dashboard functionality

Here's a quick explanation of our choices:

What do you want to make? This prompt offers several possible scaffolding options. In our case, the Express API was the perfect solution since that's what we were migrating.

What do you want to call this project? You should put whatever you want here. It'll name the directory and define the naming schema for the resources you deploy to AWS.

What org do you want to add this service to? This question assumes you are using the serverless.com dashboard for managing your deployments. We're choosing to use Github Actions and AWS tooling directly though, so we've opted out of this option.

Do you want to deploy your project? This will attempt to deploy your application immediately after scaffolding. If you don't have your AWS credentials configured correctly, this will use your default profile. We needed a custom profile configuration since we have several projects on different AWS accounts so we opted out of the default deploy.

Serverless Init Output

The init script from above outputs the following:

• .gitignore
• handler.js
• package.json
• README.md
• serverless.yml

The key here is the serverless.yml and handler.js files that are outputted.

serverless.yml

service: example-serverless-express
frameworkVersion: '2 || 3'

provider:
  name: aws
  runtime: nodejs12.x
  lambdaHashingVersion: '20201221'

functions:
  api:
    handler: handler.handler
    events:
      - httpApi: '*'

handler.js

const serverless = require("serverless-http");
const express = require("express");
const app = express();

app.get("/", (req, res, next) => {
  return res.status(200).json({
    message: "Hello from root!",
  });
});

app.get("/hello", (req, res, next) => {
  return res.status(200).json({
    message: "Hello from path!",
  });
});

app.use((req, res, next) => {
  return res.status(404).json({
    error: "Not Found",
  });
});

module.exports.handler = serverless(app);

As you can see, this gives a standard Express server ready to just work out of the box. However, we needed to make some quality of life changes to help us migrate with confidence, and allow us to use our API locally for development.

Quality of Life Improvements

There are several things that Serverless Framework doesn't provide out of the box that we needed to help our development process. Fortunately, there are great plugins we were able to install and configure quickly.

Environment Variables

We need per-environment variables as our OAuth providers are specific per host domain. Serverless Framework supports .env files out of the box but it does require you to install the dotenv package and to turn on the useDotenv flag in the serverless.yml.

Babel/TypeScript Support

As you can see in the above handler.js file, we're getting CommonJS instead of modern JavaScript or TypeScript. To get these, you need webpack or some other bundler. serverless-webpack exists if you want full control over your ecosystem, but there is also serverless-bundle that gives you a set of reasonable defaults on webpack 4 out of the box. We opted into this option to get us started quickly.

Offline Mode

With classic Express servers, you can use a simple node script to get the server up and running to test locally. Serverless wants to be run in the AWS ecosystem making it. Lucky for us, David Hérault has built and continues to maintain serverless-offline allowing us to emulate our functions locally before we deploy.

Final Configuration

Given these changes, our serverless.yml file now looks as follows:

service: starter-dev-backend
frameworkVersion: '2 || 3'
useDotenv: true # enable .env file support

plugins: # install serverless plugins
  - serverless-bundle
  - serverless-offline

custom: # configure serverless-offline
  serverless-offline:
    httpPort: 4000

provider:
  name: aws
  profile: exampleprofile # use your own profile
  stage: production # use your specified stage
  runtime: nodejs14.x
  lambdaHashingVersion: '20201221'

functions:
  api:
    handler: handler.handler
    events:
      - httpApi: '*'

Some important things to note:

• The order of serverless-bundle and serverless-offline in the plugins is critically important.
• The custom port for serverless-offline can be any unused port. Keep in mind what port your frontend server is using when setting this value for local development.
• We set the profile and stage in our provider configuration. This allowed us to use specify the environment settings and AWS profile credentials to use for our deployment.

With all this set, we're now ready to deploy the basic API.

Deploying the new API

Serverless deployment is very simple. We can run the following command in the project directory:

$ serverless deploy

This command will deploy the API to AWS, and create the necessary resources including the API Gateway and related Lambdas. The first deploy will take roughly 5 minutes, and each subsequent deply will only take a minute or two! In its output, you'll receive a bunch of information about the deployment, including the deployed URL that will look like:

https://<hash-string>.execute-api.<region>.amazonaws.com

You can now point your app at this API and start using it.

Next Steps

A few issues we still have to resolve but are easily fixed:

• New Lambdas are not deploying with their Environment Variables, and have to be set via the AWS console. We're just missing some minor configuration in our serverless.yml.
• Our deploys don't deploy on merges to main. For this though, we can just use the official Serverless Github Action. Alternatively, we could purchase a license to the Serverless Dashboard, but this option is a bit more expensive, and we're not using all of its features on this project. However, we've used this on other client projects, and it really helped us manage and monitor our deployments.

Conclusion

Given all the above steps, we were able to get our API up and running in a few minutes. And due to it being a 1-to-1 replacement for an existing Express server, we were able to port our existing implementations into this new Serverless implementation, deploy it to AWS, and start using it in just a couple of hours.

This particular architecture is a great means for bootstraping a new project, but it does come with some scaling issues for larger projects. As such, we do not recommend a pure Serverless, and Express for monolithic projects, and instead suggest utilizing some of the amazing capabilities of Serverless Framework with AWS to horizontally scale your application into smaller Lambda functions.

This Dot Labs is a development consultancy focused on providing staff augmentation, architectural guidance, and consulting to companies.

We help implement and teach modern web best practices with technologies such as React, Angular, Vue, Web Components, GraphQL, Node, and more.