szymon-szym for AWS Community Builders

Posted on Oct 16

AWS Lambda with Go, initial boilerplate

#go #aws #serverless

Introduction

Go has always been one of my favorite languages due to its simplicity. Lately, I decided to figure out what it takes to create a simple boilerplate serverless project with lambda functions written in Go. I was curious about the tooling and developer experience.

Goal

I want to create a REST API, that uses postgres db as a data layer. My initial requirements are the following

Spec to be defined with OpenApi, and models generated from it
Use AWS SAM
Each endpoint is to be handled by a separate lambda function
Local development as easy as possible
Same for deployment

Prerequisites

You would need to have Go installed, as well as AWS SAM. If you deploy your project to AWS you might be billed for the resources you are creating, so remember to clear your resources when you don't need them.
For DB I use supabase

Building the project

The code is available in this repo

Let's start by running sam init. I picked the Hello World template, Go with provided al.2023 env. Previously there was a managed runtime for Go, but nowadays it is deprecated.

OpenApi schema

Having API schema defined as OpenApi spec has a few obvious advantages. We can use it to generate documentation, create clients, servers, etc. I also use it for defining the shape of the AWS HttpApi Gateway.

My schema is straightforward. The only interesting part is x-amazon-apigateway-integration property, which allows connection with lambda integration. The setup is language-agnostic.

You can find schema file in the repo

SAM template

HttpApi and secret

# template.yaml
# ....
ItemsAPI:
    Type: AWS::Serverless::HttpApi
    Properties:
      StageName: Prod
      DefinitionBody:
        'Fn::Transform':
          Name: 'AWS::Include'
          Parameters:
            Location: './api.yaml'
      FailOnWarnings: false
DBSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: my-db-secret
      Description: Postgres config string
      SecretString: 'host=172.17.0.1 port=5431 user=admin password=root dbname=lambdas sslmode=disable'
# ....

As mentioned above, there is nothing specific to Go here. The HttpApi Gateway is created based on OpenApi.

There is also a secret for storing connection string. I will update its value after the deployment

Lambda Function

AWS SAM support for Go is pretty awesome. I can point the CodeUri to the folder with lambda handler and define the build method as go1.x

Lambda functions built in Go use provided.al2023 runtime, as they produce a single self-containing binary.

The definition of the function looks like this:

# template.yaml
# ....
  GetItemFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x
    Properties:
      Tracing: Active
      CodeUri: lambda_handlers/cmd/get_item/
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures:
        - x86_64
      Environment:
        Variables:
          DB_SECRET_NAME: !Ref DBSecret
          API_STAGE: Prod
      Events:
        HttpApiEvents:
          Type: HttpApi
          Properties:
            Path: /item/{id}
            Method: GET
            ApiId: !Ref ItemsAPI
      Policies: 
        - AWSLambdaBasicExecutionRole
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref DBSecret
# ....

Thanks to the SAM magic, the connection between HttpApi Gateway and the lambda function will be established with all required permissions.

Function code

Project structure

To be honest, the folder structure is probably not idiomatic. But I tried to follow general Go patterns

lambda_handlers
|--/api
|--/cmd
|--/internal
|--/tools
|--go.mod
|--go.sum

cmd is the main folder with actual lambda handlers
internal holds the code shared between handlers
tools defines additional tools to be used in the projects
api for openapi generator config and generated models

Function handler

The initial boilerplate for the lambda handler looks like this:

// ...
func handleRequest(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    // handler logic
}

func main() {
    lambda.Start(handleRequest)
}

Usually, the first question to ask is where to put the initialization of AWS SDK clients, database connection, and other things, we want to deal with during the cold start.

We have options here. First is to follow the pattern from AWS documentation example and initialize services inside the init() function. I don't like this approach, because it makes it harder to use the handler in the unit tests.

Thanks to the fact that lambda.Start() method takes a function as an input, I can wrap it in the custom struct, and initialize it with the services I need. In my case, the code looks this way:

package main

// imports ...

type GetItemsService interface {
    GetItem(id int) (*api.Item, error)
}

type LambdaHandler struct {
    svc GetItemsService
}

func InitializeLambdaHandler(svc GetItemsService) *LambdaHandler {
    return &LambdaHandler{
        svc: svc,
    }
}

func (h *LambdaHandler) HandleRequest(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {

    strId, err := helpers.GetPathParameter("id", request.PathParameters)

    if err != nil {
        return helpers.SendResponse("id is required", 400), nil
    }

    id, err := strconv.Atoi(*strId)

    if err != nil {
        return helpers.SendResponse("id must be an integer", 400), nil
    }

    log.Printf("id: %d", id)

    result, err := h.svc.GetItem(id)

    if err != nil {
        if err.Error() == "Item not found" {
            return helpers.SendResponse("Item not found", 404), nil
        }
        return helpers.SendResponse("error", 500), nil
    }

    jsonRes, err := json.Marshal(result)

    if err != nil {
        log.Printf("error marshalling response: %s", err.Error())
        return helpers.SendResponse("internal server error", 500), nil
    }

    return helpers.SendResponse(string(jsonRes), 200), nil
}

func main() {

    dbSecretName := os.Getenv("DB_SECRET_NAME")

    log.Printf("dbSecretName: %s", dbSecretName)

    cfg, err := awssdkconfig.InitializeSdkConfig()

    if err != nil {
        log.Fatal(err)
    }

    secretsClient := awssdkconfig.InitializeSecretsManager(cfg)

    connString, err := secretsClient.GetSecret(dbSecretName)

    if err != nil {
        log.Fatal(err)
    }

    conn, err := db.InitializeDB(connString)

    if err != nil {
        log.Fatal(err)
    }

    defer conn.Close()

    log.Println("successfully connected to db")

    svc := items.InitializeItemsService(conn)

    handler := InitializeLambdaHandler(svc)

    lambda.Start(handler.HandleRequest)
}

In the main() function (which runs during cold start) I get the secret from secretsmanager and then initialize the connection with DB. Both functionalities are defined inside internal folders as common helpers so they can be reused in other handlers. Finally my ItemsService is initialized with the created db connection, and used for creating a lambda handler.

HandleRequest parses ID from the path parameter, and calls ItemsService to get an item from DB.

Internal modules

As the function is simple, there is not much business logic around. The ItemsServise simply calls db for the specific item

package items

import (
    "database/sql"
    "errors"
    api "lambda_handlers/api"
    "log"
)

type ItemsService struct {
    conn *sql.DB
}

func InitializeItemsService(conn *sql.DB) *ItemsService {
    return &ItemsService{
        conn: conn,
    }
}

func (svc ItemsService) GetItem(id int) (*api.Item, error) {

    log.Printf("Getting item id %v", id)

    query := `SELECT * FROM items WHERE id = $1`

    var item api.Item

    err := svc.conn.QueryRow(query, id).Scan(&item.Id, &item.Name, &item.Price)

    log.Printf("Got item id %v", id)

    if err != nil {
        log.Printf("Error getting item %v: %v", id, err)
        if err == sql.ErrNoRows {
            return nil, errors.New("Item not found")
        }
        return nil, err
    }

    return &item, nil

}

At this point, we don't need anything more here.

Tools

My goal is to use additional tools, which could be attached to the project dependencies, so there is no need to rely on tools installed on the developer's machine.

In Go one way of doing it is to keep oapi-codegen in the tools package

//go:build tools
// +build tools

package main

import (
    _ "github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen"
)

And call it from inside api_gen.go

//go:generate go run github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen --config=cfg.yaml ../../api.yaml

package api

This way I can run go generate without installing oapi-codegen binaries separately.

Local development

Build

My build process requires two steps: generating models from OpenAPI, and building the projects itself. I let AWS SAM deal with the latter.

Here is my Makefile

.PHONY: build local deploy

generate:
    cd lambda_handlers && go generate ./...

build: generate
    rm -rf .aws-sam
    sam build

local: build
    sam local start-api --env-vars parameters.json

deploy: build
    sam deploy

Initial deployment

For me, the easiest way to test API Gateway locally is to run sam local start-api
Since our function relies on environment variables, I created paramters.json file to pass env vars to sam local

When developing for serverless, at some point you might want to start using cloud resources even for local development. In my case, I will utilize the secrets manager right away to store the connection string for DB. It means, that I need to deploy the stack first, so I can use it in the local development.

I run make deploy but for now I don't check the whole deployment, just grab a secret name from the console. I also need to update the secret in the console, so it holds the correct connection string.

For testing, I have created a DB on supabase and seeded it with a few dummy records

Local testing

After running make local I can test API locally

Since Go is a compiled language, after each change I need to rebuild the project and run start-api again. Considering the Go compiler's amazing speed, it is not a big deal.

Test on AWS

The API Gateway URL was printed out in the console after deployment, and it can be also grabbed from the AWS console directly.

I call the endpoint, and it works as expected:

Cold start is a bit long, as initialization takes ~300 ms, mostly because it includes taking the secret and establishing a connection to the DB. But to be honest it is more than a decent result.

Summary

The given project is a starting point for creating a serverless REST API in Go. It uses OpenAPI for the schema and AWS SAM for managing deployment and local testing.

I've used an external postgres db and AWS SDK for getting connection string from secrets manager.

There are also unit tests for lambda handler and items service

Most of the time I've spent configuring the AWS part, which would be the same for all languages. The Go code is pretty straightforward (for this simple use case).

DEV Community