Fully Serverless DERN Stack TODO App Pt. 1 - (DynamoDB, Express, React, Node)

Pt. 1 - Setting up our Backend API and deploying to AWS

Update 3/2/2022 Pt. 2 is now published.

Completed Pt.1 Github Repo

Sorry for doing a boring TODO app, I figured there were enough moving parts with this write-up between, Express, React, AWS, Serverless, etc that making a very simple application would be welcomed. I'm also assuming that for this tutorial you already have some basic experience with AWS, AWS CLI, Express.js & Node.js but I'll try to make everything as beginner friendly as I can.

The MERN stack (MongoDB, Express, React, Node.js) is one of the most popular stacks among Node.js developers. However, this stack has a major achilles heel.

It requires servers *shudders*.

Even if you do deploy your code to the cloud via a FaaS (Functions as a Service) platform, that pesky M in the MERN stack, aka MongoDB needs a to be backed by a server. Either self-hosted, ie. via an EC2 instance running on AWS, or via a managed service, like MongoDB Atlas (which, also runs their instances on AWS EC2 but it has a very nice interface.)

What if, we could build a truly serverless Express.js API, with a React SPA Frontend?

Well, now we can.

AWS offers DynamoDB, a managed NoSQL database that can offer blazing-fast single-digit millisecond performance.

Additionally, the node.js library Dynamoose is a modeling tool for DynamoDB that is very similar to the highly popular Mongoose for MongoDB. Developers already familiar with the MERN stack should feel right at home using Dynamoose with minimal modifications.

Plus, with a little deployment magic help from Claudia.js, we have a very easy way to build and deploy serverless Express.js apps.

Finally, we'll build out a React SPA frontend, and deploy that on AWS Cloudfront so that we're getting the benefits of having our static code and assets delivered via a global CDN.

Side Note: I'm really playing up the "negatives" of servers & databases for dramatic effect. Servers actually aren't that big and scary. In the real-world, the backend needs of every application will obvioiusly vary greatly. Serverless is a great tool to have in the toolbelt, but I don't believe it should be the end-all be-all for every situation.

Getting Started

Let's start by setting up our project directory. I'm going to start by making my project directory called dern-todo, then inside that directory I'm also going to create a directory called backend.

mkdir dern-todo && cd dern-todo
mkdir backend && cd backend

We're going to keep all of our Express.js / Claudia.js code inside the /backend directory, and when we eventually create a React frontend SPA, it will live-in, unsurpisingly, a directory called frontend.

Make sure you're in the backend directory, then initialize our backend application with NPM init.

npm init

I'm going to use all the NPM defaults except for 2 things. 1.) I'm changing the package name to dern-backend instead of just backend, which is pulled in from the directory name.

2.) I'm going to change "entry point: (index.js)" to app.js, which is what we'll use for our Claudia.js setup

❯ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help init` for definitive documentation on these fields
and exactly what they do.

Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
package name: (backend) dern-backend
version: (1.0.0) 
description: 
entry point: (index.js) app.js
test command: 
git repository: 
keywords: 
author: 
license: (ISC) 
About to write to /Users/[path]/dern-todo/backend/package.json:

{
  "name": "dern-backend",
  "version": "1.0.0",
  "description": "",
  "main": "app.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}


Is this OK? (yes) 

From our /backend directory, let's go ahead and install express. We'll also install nodemon and save it as a dev-depency to automatically restart our server on code changes.

npm install express
npm install --save-dev nodemon

Next, house-keeping item, I like to put all code assets into a /src directory to help keep things organized.

Then, after we create that directory, we'll also create our app.js file, PLUS an app.local.js which we'll use to run our app locally while testing.

mkdir src && cd src
touch app.js
touch app.local.js

Now we'll setup a very simple express to get everything setup for further development.

Thanks to attacomsian for a great Claudia.js setup which I'm basing the Claudia.js portion of this write-up on.

backend/src/app.js

const express = require('express')
const app = express()

app.get('/', (req, res) => res.send('Hello world!'))

module.exports = app;

Then, our app.local.js file

backend/src/app.local.js

const app = require('./app')
const port = process.env.PORT || 3000

app.listen(port, () => 
  console.log(`App is listening on port ${port}.`)
)

Finally, edit backend/package.json to add the following script:

{
  "name": "dern-backend",
  ...
  "scripts": {
    "dev": "nodemon src/app.local.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  ...
}

We can confirm that our express app works by running the following command:

npm run dev

You should see the following output:

❯ npm run dev

> dern-backend@1.0.0 dev
> nodemon src/app.local.js

[nodemon] 2.0.15
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node src/app.local.js`
App is listening on port 3000.

With that up and running, let's get the Claudia.js stuff configured so we can deploy our app to AWS. First, you can check if you already have Claudia installed on your system by running:

claudia --version

If you see a version number returned, ie 5.14.0, you're all set. If not, you can install Claudia.js globally with the following command:

npm install -g claudia

Note that we're using the -g flag with NPM to install the claudia package globally.

After that completes, you can confirm the installation was successful by running the above claudia --version command.

With Claudia, successfully installed, we're ready to use it to generate AWS Lambda wrapper. Run the following command from the /backend directory:

claudia generate-serverless-express-proxy --express-module src/app

You should see the following output in the terminal:

❯ claudia generate-serverless-express-proxy --express-module src/app
npm install aws-serverless-express -S

added 3 packages, and audited 171 packages in 2s

18 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
{
  "lambda-handler": "lambda.handler"
}

Note that in our backend directory, a new file lambda.js has been created. This file has configuration values for claudia.

With that in-place, we're almost ready to do an initial deploy to AWS. We'll just need to make sure we've configured the AWS CLI & credentials.

Sure, at the moment our express "app" is just a simple "Hello, World!", but let's make sure we're deploying early & often so we can work out any bugs / differences between local and AWS.

Run the following command:

claudia create --handler lambda.handler --deploy-proxy-api --region us-east-1

This will take a little bit of time to run, as claudia is doing some important stuff for us automatically, but you should see status updates in your terminal. Once it completes, you should see a json output with some information about our claudia app.

saving configuration
{
  "lambda": {
    "role": "dern-backend-executor",
    "name": "dern-backend",
    "region": "us-east-1"
  },
  "api": {
    "id": "[api-id]",
    "url": "https://[api-id].execute-api.us-east-1.amazonaws.com/latest"
  }
}

If you're unfamiliar with AWS services such as Lambda & API Gateway, I'll briefly explain. Lambda is AWS's "Functions As A Service" platform, it allows to upload code (in our case node.js code) and run it on-demand, as opposed to needed to deploy, provision and manage node.js servers.

There's a variety of ways you can invoke your Lambda function once it's uploaded onto AWS, but the method we're going to be using (through Claudia.js that is), is via an API Gateway.

API Gateway is a service that allows to deploy API's on AWS. One of the ways API Gateway works is by allowing you to specify various endpoints, and invoke specific Lambda functions when a request is made to that endpoint.

Although manually defining endpoints in AWS can be a useful method to build and deploy micro-services, Cluadia.js allows us to deploy our express app as a single Lambda function, and uses a proxy resource with a greedy path variable to pass the endpoints to our express app.

Below is what you'll see in the AWS Console for API Gateway after Claudia finishes deploying.

I won't go into too much detail here about the various settings and configurations of API Gateway, but the laymans version of how to interpret the image above is that API Gateway will pass any HTTP requests, ie. POST /api/auth/login {"user":"username":"pass":"password"} (that's just psuedo-code), to our Lambda function, which is an Express.js app, and the Express.js App Lambda function will handle the request the same way it would if the app were running on a server.

If that sounds complicated, don't worry, we'll run through a quick example to see how everything is working.

Throughout the rest of this write-up / series, I'm going to be using Postman to test out our api until we build out a frontend. I'm going to keep all related requests in a Postman collection named "Serverless DERN TODO". Going into too much detail on Postman is going to be outside the scope of this tutorial, but I'll try to explain what I'm doing each step of the way in case this is your first time using the tool.

If you'll recall back to our app.js file from earlier, you'll remember that we setup a single GET endpoint at our API root. Let's use Postman to make a GET request there and confirm everything is working.

The URL that we will make the request to is the url from the Claudia json output earlier:

{
  "lambda": {
    ...
  },
  "api": {
    "id": "[api-id]",
    "url": "https://[api-id].execute-api.us-east-1.amazonaws.com/latest" <- This thing
  }
}

If you need to find that information again, you can either go into the AWS API Gateway console, click "Stages", then "latest". The URL is the "Invoke URL".

Or, you'll notice that after we ran the claudia create ... command earlier, a new claudia.json file was created which stores our api-id & the region we deployed our api to, in this case us-east-1. You can take those two values and put them into the following URL pattern

https://[api-id].execute-api.[aws-region].amazonaws.com/latest

Note: The /latest path at the end of our Invoke URL is the "stage" from API Gateway. You can configure multiple stages (ie dev, v1, etc) but the default stage Claudia creates for us is "latest". Express will start routing after the /latest stage. For example, if we made a /login endpoint, the final URL would look like https://[api-id].execute-api.[aws-region].amazonaws.com/latest/login

Here's our Postman GET request to the API root. We get back, Hello world!

Don't forget, we also setup our app.local.js file so that we can develop and test on our local machine. Run the npm dev command to startup our express app.

npm run dev

> dern-backend@1.0.0 dev
> nodemon src/app.local.js

[nodemon] 2.0.15
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node src/app.local.js`
App is listening on port 3000.

I'm also going to change our Base URL to a Postman variable. Highlight, the entire url in our request, click the "Set as variable" popup that appears, then select, "Set as a new variable". I'm naming my variable BASE_URL and setting the scope to the collection. Finally, click the orange "Set variable" button to save.

If all went correctly, you should see the url in the GET request changed to {{BASE_URL}}.

Now that we've promoted our API Gateway URL to a variable, it's time to immediately change its value to point to our localhost server.

Access the variables by clicking on the name of the collection in the left-hand sidebar (mine is named Serverless DERN TODO). Then click the "variables" tab, you should see BASE_URL the variable we just created. It has two fields, "INITIAL VALUE" & "CURRENT VALUE". Change the URL inside "CURRENT VALUE" to "http://localhost:3000".

IMPORTANT! Don't forget to save BOTH the collection and the GET request to ensure that Postman is using the updated value for the variable. The orange circles on the request and collection tabs will let you know if you have unsaved changes.

You should be able to send the GET request again, and see the same Hello world! response. Right now, we don't have any logging in our app, so you won't see anything in the terminal running our local version of the app. The only difference you might notice is a significantly lower ms response time vs. the AWS API Gateway request, since our localhost version doesn't have very far to go.

With all that setup, we're in a good place to stop for Part 1. We've accomplished a lot so far, we have an Express.js app setup and ready to easily deploy to AWS via Claudia.js. We also have a local dev version of our Express app ready for further development and testing.

Up next is Pt. 2 of the series where we'll start foucsing on building out the features of our application like building some data models with Dynamoose.