Learn what is needed to build a Lambdaless API using AWS API Gateway, DynamoDB, OpenAPI and CloudFormation.
A common way people build serverless APIs is by routing an API Gateway request to an AWS Lambda. This will make another request to a different AWS Service. It often goes unnoticed that API Gateway can integrate with other AWS Services without the need of Lambda.
Most of the operations that we do in Lambda to fulfill a request are:
- Gather the information from the input body.
- Map the input from one service to another.
- Map the service's output to what we want to return to the client.
You will see time and time again that you are doing these 3 steps over and over again.
This process is possible without using a Lambda to handle the mappings. API Gateway lets you use Apache's Velocity Templating Language (VTL) to do what the Lambda did. This will reduce latency and cost by removing the Lambda invocations from each call.
This article will show you what you need to be able to do this without going into the AWS Console. By avoiding the AWS Console we make sure that the code is ready to be integrated into a CI/CD pipeline.
Let's get started
The example shows how to create a Products Service that will Create, Read, Update and Delete (CRUD) products from the database.
File Structure
The solution only requires two files:
template.yaml - Defines the DynamoDB table, API Gateway and the role that API Gateway will use to execute. CloudFormation uses this file to deploy all our resources.
products-openapi.yaml - this is where all the magic happens. This file has the API definition and the mappings needed for the input and output by using VTL.
Define resources using SAM template
I am using the Serverless Appliction Model(SAM). SAM is a tool that lets you define your Infrastructure as Code. SAM reads from the template.yaml file and runs it through CloudFormation. CloudFormation then creates all the resources in the AWS Cloud.
The resources needed for the service to work are the following:
- DynamoDB table - Defines a single table with a Partition Key named pk. The value of this attribute will be a GUID. (For more complex data models I would recommend watching this video by Rick Houlihan)
ProductsTable:
Type: AWS::DynamoDB::Table
Properties:
KeySchema:
- AttributeName: pk
KeyType: HASH
AttributeDefinitions:
- AttributeName: pk
AttributeType: S
BillingMode: PAY_PER_REQUEST
- API Gateway - Loads the API definition from the OAS file. It is using a Transform to be able to use intrinsic functions to get things like ARNs, Parameters, etc.
ProductsAPI:
Type: AWS::Serverless::Api
Properties:
StageName: Lambdaless
DefinitionBody:
'Fn::Transform':
Name: AWS::Include
Parameters:
Location: ./products-openapi.yaml
- Role - Role that gives API Gateway the permissions needed to be able to run DynamoDB actions.
ProductsAPIRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- sts:AssumeRole
Policies:
-
PolicyName: ProductsAPIPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- "dynamodb:PutItem"
- "dynamodb:UpdateItem"
- "dynamodb:DeleteItem"
- "dynamodb:GetItem"
- "dynamodb:Scan"
Resource: !GetAtt ProductsTable.Arn
Create API Gateway endpoints using Open API
A simple way to define your API endpoints is by using the Open API Specification (OAS). OAS has become the industry standard for API definitions. There are a lot of tools that read the file and generate documentation. Postman and SwaggerHub are some of the applications that do this.
CloudFormation will create the API by using what you already defined using OpenAPI. You have to use Amazon's custom OAS extensions, to be able to set your VTL I/O mappings.
This Open API Specification example shows the definition of 5 RESTful endpoints. I will be highlighting several parts of this file to better understand what is happening.
First you have the paths, each path will then have each of the operations that it will run: GET, POST, PUT, DELETE.
In the example API we have two paths:
/products
This defines two endpoints:
- GET - Get list of all the products.
- POST - Create new Product.
/products/{productId}
This path takes in a productId to work on a specific product. The endpoints that we use this path for are:
- GET - Get products details.
- PUT - Update product item with the input information.
- DELETE - Delete product.
There are three sections under each of the endpoints:
- requestBody - this section only applies to endpoints that take in an input (POST and PUT in the example). It defines the expected input model. API Gateway validates that the input has expected model. If the validations fail it will return a 400 Bad Request.
- responses - list of the expected responses when calling the endpoint.
- x-amazon-apigateway-integration - This is a custom AWS extension. It defines the mappings for the inputs and outputs for the call to DynamoDB. We will go deeper into this section below.
Breaking down x-amazon-apigateway-integration extension
The example below shows how the endpoint that adds products implements this custom extension.
x-amazon-apigateway-integration:
httpMethod: POST
type: AWS
uri: { "Fn::Sub": "arn:aws:apigateway:${AWS::Region}:dynamodb:action/PutItem" }
credentials: { "Fn::GetAtt": [ ProductsAPIRole, Arn ] }
requestTemplates:
application/json:
Fn::Sub:
- |-
{
"TableName": "${tableName}",
"Item": {
"pk": {
"S": "$context.requestId"
},
"productname": {
"S": "$input.path("$.name")"
}
}
}
- { tableName: { Ref: ProductsTable } }
responses:
default:
statusCode: 201
responseTemplates:
application/json: '#set($inputRoot = $input.path("$"))
{
"id": "$context.requestId"
}'
There are a lot more properties that can be set for this integration, for a full list go here. I will be highlighting a few that are being used in the example:
uri - Endpoint that API Gateway will be calling to execute our action. In the example it's calling the DynamoDB Put Item endpoint.
credentials - It uses the role that was defined in the * template.yaml. It's using the GetAtt intrinsic function to get the Roles ARN.
requestTemplates - This takes the body of the request and maps it to the DynamoDB input using VTL. The first item of the array is using the requestId to set the ProductId. To see a list of all the variables available in the $context go here. This also makes use of the $input variable to get the values from the input, the example is using it to get the name attribute. In the second item we use the Fn::Sub intrinsic function to get the Table Name defined in the template.yaml.
responses - Contains the responseTemplates. This is using the same $input and $context variables to build the response mapping.
In the next example you can see how to use a foreach expression using VTL. This mapping is creating a products array as the response of the API endpoint.
responseTemplates:
application/json: '#set($inputRoot = $input.path("$"))
{
"products": [
#foreach($elem in $inputRoot.Items) {
"id": "$elem.pk.S",
"name": "$elem.productname.S",
}#if($foreach.hasNext),#end
#end
]
}'
There are a lot more things that can be done with Open API and Amazon API Gateway Extensions. To get more information on how you can use these go to these links:
Deploy the Service to AWS
As I mentioned before this is all done without touching the AWS console. To get the application deployed to AWS there are a few things that need to happen:
1. SAM needs to build the artifacts that are going to be deployed.. This is done by running the following SAM command:
sam build
This creates a folder named .aws-sam that contains the built
artifacts.
2. With the artifacts build SAM can deploy them to the cloud.
As of version v0.33.1 the sam cli introduced the capability to
deploy using a samconfig.toml file. You can have the sam cli
generate this file by running sam deploy in guided mode.
To create the samconfig.toml file you need to run the following command:
sam deploy --guided
This will ask you a few simple questions to be able to populate
the config file. Everything will be available to test once the
script is done.
How to test the API
To test this we need the deployed APIs URL.
For security reasons the APIs URL is not accessible by calling an endpoint, which means we need to build it ourselves with information we already have in the template. We are going to join several values to build the URL and output it.
It ends up looking like this:
Outputs:
ProductsURL:
Description: Products URL
Value: !Join [ '', [ 'https://', !Ref ProductsAPI, '.execute-api.', !Sub '${AWS::Region}', '.amazonaws.com/Lambdaless' ] ]
Whenever you run a deployment you will be able to get this value from the terminal.
You can use a tool like Postman to make requests to your API. All you have to do is append the path as defined in the OAS file as well as the necessary parameters and body.
I included a Postman collection in the example that can be imported to view how the requests look.
Bonus
To simplify deployments I included a package.json file. The file has an npm script that will take care of executing the deployment commands.
The script looks like this:
"deploy": "sam build && sam deploy"
Now you can use npm run to execute the deployment:
npm run deploy
By now you should understand how to build and deploy a Lambdaless REST API. As a bonus you get self documenting APIs by using OAS with Postman.
I hope you have enjoyed this. You can find a link to the full example here
Top comments (0)