AWS project - Building a serverless microservice with Lambda

Overview

In this article, we are going to build a simple serverless microservice on AWS that enables users to create and manage movies data. We will set up an entire backend using API Gateway, Lambda and DynamoDB. Then we will use SwaggerHub as an integrated design environment for our APIs:

The goal of this project is to learn how to create an application composed of small, easily deployable, loosely coupled, independently scalable, serverless components.

As I’m strongly against managing environments manually and take Infrastructure as Code for granted (if we are not on the same page, I’d suggest you to read this article first), AWS SAM will be a great fit for our serverless application. As a result, the entire application should be deployed in any AWS account with a single CloudFormation template.

Architecture

1️⃣ A user requests to the server by calling APIs from SwaggerHub UI. User's request which includes all necessary information is sent to Amazon API Gateway restful service.
2️⃣ API Gateway transfers the collected user information to a particular AWS Lambda function based on user's request.
3️⃣ AWS Lambda function executes event-based logic calling DynamoDB database.
4️⃣ DynamoDB provides a persistence layer where data can be stored/retrieved by the API's Lambda function.

The high-level architecture for the serverless microservice is illustrated in the diagram below:

Source code

All source code for this project is available on GitHub in a public AwsServerlessMicroserviceWithLambda repository.

To clone the GitHub repository, execute the following command:

cd my-folder
git clone https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda.git

Initial Setup

To codify, build, package, deploy, and manage our AWS resources in a fully automated fashion, we will use:

👉 AWS SAM
👉 AWS Cloud​Formation
👉 AWS CLI
👉 AWS SDK for Python (boto3)
👉 Docker

If you don't want to install or maintain a local IDE, use AWS Cloud9 instead (you can find how to set up AWS Cloud9 here). Otherwise you might need to install the latest AWS CLI, AWS SAM and Python on your development machine.

AWS Resources

Here is the list of AWS resources that we are going to create:

✔️ AWS Lambda
✔️ Amazon DynamoDB
✔️ Amazon API Gateway
✔️ AWS IAM
✔️ Amazon S3 (that is where your CloudFormation template will be stored)

In this project, we will be building Lambda functions with Python 3.8.

Step 1. Generate an AWS SAM template

Download AWS SAM Hello World template which implements a basic API backend based on Amazon API Gateway endpoint and AWS Lambda function:

cd my-folder
sam init

If you're feeling lost, please, follow Step 2. Create a SAM application described in my previous article.

AWS SAM should create a directory with all necessary files and folders:

Step 2. Create a DynamoDb table

Rewrite pre-generated "template.yaml" with the following code:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: 'SAM Template for ServerlessMicroserviceApp'

# ======================== RESOURCES ======================== #
Resources:
  MoviesTable:
    Type: AWS::Serverless::SimpleTable
    Properties:
      TableName: 'movies-table'
      PrimaryKey:
        Name: uuid
        Type: String

AWS::Serverless::SimpleTable is AWS SAM syntax for DynamoDB table creation with a single attribute primary key named uuid. Note, if you want to add sort key or use other advanced functionality of DynamoDB, then you may want to consider using AWS CloudFormations's AWS::DynamoDB::Table resource instead (yes, you can use AWS CloudFormation syntax in AWS SAM template, combining the best of both worlds 💙).

Step 3. Create an API Gateway

Next step is to define API Gateway in "template.yaml":

MoviesApi: 
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      # DefinitionUri: ./swagger.json

AWS::Serverless::Api is AWS SAM syntax for a collection of Amazon API Gateway resources. FYI, there is no need to explicitly add Api to the template - AWS SAM can implicitly create it based on fromRestApiId attribute defined on AWS::Serverless::Function resources (see Step 4 below). However I prefer an explicit definition as AWS::Serverless::Api has a lot of useful properties I might use in the future (without actially refactoring the template).

Amazon API Gateway acts as the interface layer between the frontend (SwaggerHub) and AWS Lambda, which calls the backend (DynamoDB database). Below are the different APIs that we are about to define in Events attribute of each Lambda function in Step 4:
👉 POST /movie (CreateMovie)
👉 GET /movie/{uuid} (GetMovie)
👉 GET /movie/list (GetMovies)
👉 PUT /movie (UpdateMovie)
👉 DELETE /movie/{uuid} (DeleteMovie)

Step 4. Create Lambda functions

We are going to create five lambda functions:

• CreateMovieFunction (create_movie/app.py)
  • triggered by POST /movie API
• GetMovieFunction (get_movie/app.py)
  • triggered by GET /movie/{uuid} API
• GetMoviesFunction (get_movies/app.py)
  • triggered by GET /movie/list API
• UpdateMovieFunction (update_movie/app.py)
  • triggered by PUT /movie API
• DeleteMovieFunction (delete_movie/app.py)
  • triggered by DELETE /movie/{uuid} API

Let's start with CreateMovieFunction:

1️⃣ Rename "hello_world" folder to "create_movie".

2️⃣ Clear the content of "create_movie/requirements.txt" file (no need in any Python dependency).

3️⃣ Open "create_movie/app.py" file and add the following code:

import json
import boto3
import uuid
from botocore.exceptions import ClientError

# Extract movie object from request body and insert it into DynamoDB table
def lambda_handler(event, context):

    if ('body' not in event or event['httpMethod'] != 'POST' or 'title' not in event['body']):
        return {
            'statusCode': 400,
            'body': json.dumps({'message': 'Bad Request'})
        }

    new_movie = json.loads(event['body'])
    response = add_movie_to_db(new_movie)

    return {
        'statusCode': 200,
        'body': json.dumps({'message': 'A new movie was saved successfully in database'})
    }

# Insert a new item into DynamoDB table
def add_movie_to_db(new_movie):

    new_movie['uuid'] = str(uuid.uuid1())

    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('movies-table')

    try:
        response = table.put_item(Item=new_movie)
    except ClientError as e:
        print(e.response['Error']['Message'])
    else:
        return response

This function helps to add a new item to the DynamoDB's movies-table. title field is the only required property of body object passed by a user. uuid field, as a primary key that uniquely identifies each movie, should be automatically generated by uuid.uuid1() Python function.

If you want to debug the code, please follow Step 3. Debug your function locally described in my previous article.

4️⃣ Add lambda function to "template.yaml":

# ======================== GLOBAL ======================== #
Globals:
  Function:
    Runtime: python3.8
    Handler: app.lambda_handler
    Timeout: 60 # default is 3 seconds the function can run before it is stopped
    Environment:
      Variables:
        TABLE_NAME: !Ref MoviesTable

# ======================== RESOURCES ======================== #
Resources:
  # ... other resources ...
  CreateMovieFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: 'CreateMovieFunction'
      CodeUri: create_movie/
      Policies: # follow the principle of least privilege
        - DynamoDBCrudPolicy: # AWS SAM policy
            TableName: !Ref MoviesTable
        - AWSLambdaBasicExecutionRole
      Events:
        CreateMovieApi:
          Type: Api
          Properties:
            RestApiId: !Ref MoviesApi
            Path: /movie
            Method: post

• Globals - resources that have common configurations (think of it as a variable in programming). As we are going to create four more functions with the same Runtime, Handler, Timeout and Environment values, we can put all these attributes into Global section following a good old DRY (Don't Repeat Yourself) principle.

• CodeUri - the Lambda function's path (might be Amazon S3 URI, local file path, or FunctionCode object). Here we are adding a reference to a create_movie folder. Note, in Global section's Handler attribute we have already specified app file name (without py file extension) and lambda_handler function name.

• Policies - one or more policies that will be appended to the default role (a new IAM role will be created) for a Lambda function. Here we are giving our function permission to access our DynamoDB table.

• Events - events that trigger the Lambda function. In our case it is POST /movie API event.

In the same way as above, we need to create four more Lambda functions:

• GetMovieFunction - get the code from GitHub here
• GetMoviesFunction - get the code from GitHub here
• UpdateMovieFunction - get the code from GitHub here
• DeleteMovieFunction - get the code from GitHub here
• The final version of the CloudFormation template - get the code from GitHub here

At the end we should get the following files and folders:

Step 5. Build and deploy the project to AWS Cloud

Build the project inside a Docker container:

cd my-folder
sam build --use-container

AWS SAM builds any dependencies that your application has, and copies your application source code to folders under .aws-sam/build to be zipped and uploaded to Lambda.

Then deploy your application using the following AWS SAM command:

sam deploy --guided

Please follow Step 5. Deploy the project to the AWS Cloud described in my previous article for a detailed explanation.

AWS SAM deploys the application using AWS CloudFormation. You can navigate to AWS Management Console -> CloudFormation to view the list of newly generated resources:

Step 6. SwaggerHub integration

Instead of logging into AWS Management Console and navigating from one API Gateway endpoint to another, we will use SwaggerHub to manage all your APIs in one place and to document them all.

SwaggerHub is an integrated API development platform that brings together all the core capabilities of the open source Swagger framework, along with additional advanced capabilities to build, document, manage, and deploy your APIs.

All you need is to create a free account on Swagger, then navigate to Create New -> Create New API:

Give a name to your template:

Next, modify information of Petstore APIs with our own APIs. For example, for POST /movie/{uuid} API the template should look like this one below:

swagger: '2.0'
info:
  description: |
    This is  a swagger for an AWS serverless microservice built with API Gateway, Lambda and DynamoDB.
  version: 1.0.0
  title: Swagger for AWS serverless microservice
  termsOfService: http://swagger.io/terms/
  license:
    name: Apache 2.0
    url: http://www.apache.org/licenses/LICENSE-2.0.html
tags:
- name: movie
  description: CRUD operations for movie domain
  externalDocs:
    description: Find out about the project
    url: https://github.com/Tiamatt/AWSServerlessMicroserviceWithLambda
paths:
  /movie/{uuid}:
    get:
      tags:
      - movie
      summary: Find a movie by uuid
      description: Returns a single movie
      operationId: getMovieByUuid
      produces:
      - application/json
      - application/xml
      parameters:
      - name: uuid
        in: path
        description: UUID of movie to return
        required: true
        type: string
      responses:
        200:
          description: successful operation
          schema:
            $ref: '#/definitions/Movie'
        400:
          description: Invalid UUID supplied
        404:
          description: Movie not found
definitions:
  Director:
    type: object
    properties:
      firstname:
        type: string
        example: Francis
      lastname:
        type: string
        example: Coppola
    xml:
      name: Director
  Movie:
    type: object
    required:
    - title
    - year
    properties:
      title:
        type: string
        example: Godfather
      year:
        type: integer
        format: int32
        example: 1972
      director:
        $ref: '#/definitions/Director'
      country:
        type: string
        example: United States
    xml:
      name: Movie   
externalDocs:
  description: Find out more about Swagger
  url: http://swagger.io
schemes:
 - https
# Added by API Auto Mocking Plugin
host: {my-id}.execute-api.us-east-1.amazonaws.com
basePath: /Prod

Get the final version of the Swagger template from GitHub here.

You need to replace host value with your API Gateway invoke url, which you can find in Outputs section of AWS CloudFormation Clonsole:

And don't forget to save your changes! Then click on View Documentation icon:

Step 7. Results

Finally, it's time to play with our serverless microservice using beautiful SwaggerHub UI:

🎉 Now, let's create a new movie:

Feel free to add some more movies.

🎉 Let's get a list of all movie:

Copy uuid of 'Godfather' movie for our next steps!

🎉 Get a movie by uuid:

🎉 Delete a movie by uuid:

🎉 And finally, let's update an existing movie:

Step 8. Cleanup

Use the AWS CloudFormation command to delete the stack along with all the resources it created:

aws cloudformation delete-stack --stack-name aws-serverless-microservice-app-stack

Conclusion

Congratulations on getting this far!

Now you know how to create a simple CRUD (create, read, update, delete) app, and to set the foundational services, components, and plumbing needed to get a basic AWS serverless microservice up and running!