hayao-k for AWS Community Builders

Posted on Dec 9, 2020 • Edited on Dec 12, 2020 • Originally published at hayao-k.dev

Let's Try Lambda Container Support with SAM CLI

#aws #lambda #docker #container

Introduction

Container image support for AWS Lambda was announced at AWS re:Invent 2020.

🚀 AWS Lambda now supports container images as a packaging format
https://aws.amazon.com/about-aws/whats-new/2020/12/aws-lambda-now-supports-container-images-as-a-packaging-format/

You might have thought that AWS had finally released a service like Cloud Run.
Unfortunately, it doesn't offer the ability to easily run any container image with Lambda.

This is a feature that allows you to package and deploy Lambda functions as container images. Therefore, the container image you create must implement the Lambda Runtime API and be compatible with Lambda functions.

The AWS SAM CLI supports container images in v1.13.1.

🚀 Release 1.13.1 - Support for Lambda Container Images
https://github.com/aws/aws-sam-cli/releases/tag/v1.13.1

Let's Try

As mentioned earlier, AWS SAM CLI must be v1.13.1 or higher.

$ sam --version
SAM CLI, version 1.13.1

Create project

You can create a project from a template with sam init command.
For package type questions, select 2 - image.
I have created a sample project using the base image from nodejs12.x.

$ sam init
Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1
What package type would you like to use?
        1 - Zip (artifact is a zip uploaded to S3)
        2 - Image (artifact is an image uploaded to an ECR image repository)
Package type: 2

Which base image would you like to use?
        1 - amazon/nodejs12.x-base
        2 - amazon/nodejs10.x-base
        3 - amazon/python3.8-base
        4 - amazon/python3.7-base
        5 - amazon/python3.6-base
        6 - amazon/python2.7-base
        7 - amazon/ruby2.7-base
        8 - amazon/ruby2.5-base
        9 - amazon/go1.x-base
        10 - amazon/java11-base
        11 - amazon/java8.al2-base
        12 - amazon/java8-base
        13 - amazon/dotnetcore3.1-base
        14 - amazon/dotnetcore2.1-base
Base image: 1

Project name [sam-app]:

Cloning app templates from https://github.com/aws/aws-sam-cli-app-templates

    -----------------------
    Generating application:
    -----------------------
    Name: sam-app
    Base Image: amazon/nodejs12.x-base
    Dependency Manager: npm
    Output Directory: .

    Next steps can be found in the README file at ./sam-app/README.md

The generated file is as follows
You will see that Dockerfile has been created.

$ cd sam-app
$ tree
.
├── events
│   └── event.json
├── hello-world
│   ├── app.js
│   ├── Dockerfile
│   ├── package.json
│   └── tests
│       └── unit
│           └── test-handler.js
├── README.md
└── template.yaml

The base image of Dockerfile is specified as an image provided by AWS.

FROM public.ecr.aws/lambda/nodejs:12

COPY app.js package.json ./

RUN npm install

# Command can be overwritten by providing a different command in the template directly.
CMD ["app.lambdaHandler"]

The base image provided by AWS is preloaded with the runtime and other components needed to execute Lambda functions, so you just need to add the Lambda function code and its dependencies.

If you want to make your own base image Lambda-compatible, you need to add Runtime Interface Clients (RIC), a set of software packages that implement the Lambda Runtime API, to your base image.

Let' take a look at the contents of template.yaml.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-app

  Sample SAM Template for sam-app

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      # ImageConfig:
        # Uncomment this to override command here from the Dockerfile
        # Command: ["app.lambdaHandler"]
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get
    Metadata:
      DockerTag: nodejs12.x-v1
      DockerContext: ./hello-world
      Dockerfile: Dockerfile

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

In order to package with a container image, PackageType: Image must be specified.
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-function.html#sam-function-packagetype

    Properties:
      PackageType: Image

To build a container image, use the Metadata resource attribute to declare information such as Dockerfile, Context, Tag, etc.

You can also use the DockerBuildArgs entry to specify the arguments for the build execution.
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-build.html#build-container-image

    Metadata:
      DockerTag: nodejs12.x-v1
      DockerContext: ./hello-world
      Dockerfile: Dockerfile

Build

Build the container image with the sam build command.

$ sam build
Building codeuri: . runtime: None metadata: {'DockerTag': 'nodejs12.x-v1', 'DockerContext': './hello-world', 'Dockerfile': 'Dockerfile'} functions: ['HelloWorldFunction']
Building image for HelloWorldFunction function
Setting DockerBuildArgs: {} for HelloWorldFunction function
Step 1/4 : FROM public.ecr.aws/lambda/nodejs:12
 ---> ccbddaf00c51
Step 2/4 : COPY app.js package.json ./
 ---> fb16c5342f63
Step 3/4 : RUN npm install
 ---> Running in faa9eb4d503c
npm WARN deprecated debug@3.2.6: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated mkdirp@0.5.4: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
npm notice created a lockfile as package-lock.json. You should commit this file.
added 107 packages from 544 contributors and audited 107 packages in 5.755s

16 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

 ---> 8db6f5505058
Step 4/4 : CMD ["app.lambdaHandler"]
 ---> Running in 08e14cc877aa
 ---> 271c7f34a0c7
Successfully built 271c7f34a0c7
Successfully tagged helloworldfunction:nodejs12.x-v1

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

A packaged container image was created.

$ docker image ls
REPOSITORY                     TAG                 IMAGE ID            CREATED             SIZE
helloworldfunction             nodejs12.x-v1       271c7f34a0c7        3 minutes ago       471MB

Test

To test locally the Lambda functions packaged as container images, you can use the Lambda Runtime Interface Emulator (RIE), which is a lightweight web server.

AWS-provided base image has a built-in RIE, so you can simply run the container from the built image and run the test　immediately.

$ docker run -d -p 9000:8080 helloworldfunction:nodejs12.x-v1
a510b1f8bf0fd5c0d7c6a3716ddaf96462452abcdff26285d1e8471c6e829cc8

$ curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
{"statusCode":200,"body":"{\"message\":\"hello world\"}"}

Since RIE is responsible for converting HTTP requests into JSON events and proxying them, it does not support X-Ray or other Lambda integrations.

If you use your own base image, you can run tests by adding the RIE to the image or by launching a container with a bind-mounted RIE as Entrypoint.
https://docs.aws.amazon.com/lambda/latest/dg/images-test.html#images-test-alternative

Of course, since we are using the SAM CLI this time, you can also run tests with sam local invoke.

$ sam local invoke
Invoking Container created from helloworldfunction:nodejs12.x-v1
Image was not found.
Building image.................
Skip pulling image and use local one: helloworldfunction:rapid-1.13.1.

START RequestId: 05284faf-5a20-46fc-a695-111eb3d0f085 Version: $LATEST
END RequestId: 05284faf-5a20-46fc-a695-111eb3d0f085
REPORT RequestId: 05284faf-5a20-46fc-a695-111eb3d0f085  Init Duration: 1.86 ms  Duration: 96.09 ms      Billed Duration: 100 ms Memory Size: 128 MB     Max Memory Used: 128 MB

Deploy

A repository must be created in advance to push the container image, but you can leave the image pushing to the SAM CLI!

$ aws ecr create-repository --repository-name lambda-container-test
{
    "repository": {
        "repositoryUri": "123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/lambda-container-test",
        "imageScanningConfiguration": {
            "scanOnPush": false
        },
        "registryId": "123456789012",
        "imageTagMutability": "MUTABLE",
        "repositoryArn": "arn:aws:ecr:ap-northeast-1:123456789012:repository/lambda-container-test",
        "repositoryName": "lambda-container-test",
        "createdAt": 1606926983.0
    }
}

Execute the sam deploy command.
By specifying the repository that you have just created in the Image Repository, the image will be pushed automatically during deployment.

$ sam deploy --guided

Configuring SAM deploy
======================

        Looking for config file [samconfig.toml] :  Not found

        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [sam-app]:
        AWS Region [us-east-1]: ap-northeast-1
        Image Repository []: 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/lambda-container-test
        Images that will be pushed:
          helloworldfunction:nodejs12.x-v1 to 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/lambda-container-test:helloworldfunction-271c7f34a0c7-nodejs12.x-v1

        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [y/N]: n
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]: y
        HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
        Save arguments to configuration file [Y/n]: y
        SAM configuration file [samconfig.toml]:
        SAM configuration environment [default]:

        Looking for resources needed for deployment: Found!

                Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-xxxxxxxxxxxxx
                A different default S3 bucket can be set in samconfig.toml

        Saved arguments to config file
        Running 'sam deploy' for future deployments will use the parameters saved above.
        The above parameters can be changed by modifying samconfig.toml
        Learn more about samconfig.toml syntax at
        https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
The push refers to repository [123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/lambda-container-test]
6d7acece320f: Pushed
105893862807: Pushed
3642f26c4fcb: Pushed
1807102b87b6: Pushed
120614c3628c: Pushed
0d8c48ae73f7: Pushed
e4f26f8be15f: Pushed
af6d16f2417e: Pushed
helloworldfunction-271c7f34a0c7-nodejs12.x-v1: digest: sha256:7a83998f07e54249948db91846abd9dbfecbe175181fe0876b5980a635b9e70f size: 1998


        Deploying with following values
        ===============================
        Stack name                   : sam-app
        Region                       : ap-northeast-1
        Confirm changeset            : False
        Deployment image repository  : 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/lambda-container-test
        Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-xxxxxxxxxxxxx
        Capabilities                 : ["CAPABILITY_IAM"]
        Parameter overrides          : {}
        Signing Profiles           : {}
--omitted--

Make a request to the created API Gateway and see how it works.

$ curl https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/Prod/hello/
{"message":"hello world"}

It works!
You can see in the Lambda console that the function code has been deployed as a container image.

Points of note

Even if you remove the stack, the image that was pushed to the ECR remains. (It will not be deleted automatically)
I haven't made an exact comparison yet, but it seems to take longer to initialize compared to a ZIP deployment package.
The initialization time is included in the billing duration. (Perhaps because it is treated the same as a custom runtime.)