Why you should deploy your next serverless application with Chalice and CDK 🐍

You usually develop serverless applications with the Serverless Framework using Node.js and face the challenge to develop one using Python? You’ve come to the right place.

To easily configure and deploy standard serverless patterns, you should switch to Chalice as an Infrastructure as Code serverless framework. Chalice natively integrates with CDK to deploy non-Lambda related resources. In addition, leverage MyPy to preserve type-safety while using AWS SDK. The tooling that comes with Python serverless ecosystem is at least as mature as Node.js one with packages like PynamoDB or AWS Lambda powertools.

Switching from Typescript to Python in a Serverless Architecture, a good idea?

Serverless infrastructure is increasingly used. Using serverless, you can easily build fast and scalable web applications without wasting time managing infrastructure.

At Kumo, we are strongly convinced that serverless is about to revolutionize the web. The results of the Datadog study "The State of Serverless" reinforce our opinion. It shows that the use of this technology is growing. At the beginning of 2021, Lambda functions have been invoked 3.5 times more often than two years before.

Our favorite stack leverages Typescript and the Serverless Framework to ship applications on AWS. The whole team has good knowledge about this stack and can efficiently develop new features using familiar services and code patterns. The Serverless Framework is very convenient to deploy Lambda functions.

However, the Serverless Framework is very Lambda-centric. Provisioning other managed services requires relying on vanilla Cloudformation, degrading considerably the developer experience. Bridging the Serverless Framework with CDK, the state-of-the-art provisioning engine, comes at a cost. Sebastien Billes, in his article, gives some keys to combine the two but he concedes that “it does come with quite a bit of overhead”. Frédéric Barthelet also explained recently how to link the two.

The entire team being full-stack developers, the convenience to use a single language across the whole stack influenced us in using Typescript. Datadog’s study reveals that a vast majority of Lambda users prefer Python: almost 60% of the AWS Lambda functions are developed with Python. Python is also a very good alternative as a runtime. It has similar cold start durations compared to Node.js ones. It could be particularly relevant to use Python to develop serverless web applications in which we have to manipulate large amounts of data. Let's dive into the state-of-the-art Python serverless ecosystem and find out how it compares to our classical Typescript one.

Defining 3 criteria to evaluate serverless Python

The criteria I used to evaluate the use of Python as a runtime are based on our daily development experience. I want to build a DX as convenient as the one we currently have in typescript.

🦺 Type safety of AWS tooling: At Kumo, we use typescript instead of javascript because type safety is very important to us. We develop great web applications for our clients and we want these applications to be as stable as possible. Boto3, the AWS Software Development Kit for Python, is not typed well enough by itself. MyPy is a good typing solution for Python and it includes better typing for boto3.

🚀 Ease of deployment: I explained before that one of the reasons why we like the Serverless Framework is its convenience in deploying our application. I would like my Python application to be deployed as easily regardless of the framework I choose to use. The next section explains why the Python framework Chalice, paired with CDK, satisfies our requirements.

👵 Community maturity on AWS services: The tooling provided by the community to use DynamoDB is a way to measure it. We widely use the AWS NoSQL database DynamoDB in our projects. We combine it with the node package dynamodb-toolbox which provides abstracted interfaces, additional validation, parsing, and data transformation features. PynamoDB from the Python ecosystem matches features from its Node.js counterpart.

Chalice: A Python decorator-based serverless framework

What does it look like?

Chalice is a framework for creating serverless web applications in Python. It is fully written in Python and designed only for this language.

Chalice provides a nice API to easily deploy Lambda functions. It is a decorator-based API similar to that of Flask. The decorator is used to indicate the trigger of the Lambda (HTTP route, WebSocket...). Pure Lambda functions, lambdas for which no trigger is defined, can also be created with this framework. Pure lambdas are very convenient for example when you want to integrate such lambdas in a step-function, where no decorator is in place to define such trigger.

This decorator API results in a very compact code to define the lambdas on the Chalice code. As an example, here is the code for three different lambdas with different triggers:

# Lambda triggered on HTTP request POST on route /apple
@app.route('/apple', methods=['POST'],cors=cors_config)
def create_apple_http():
    create_apple()

# Lambda triggered on websocket connection
@app.on_ws_connect()
def connect(event:WebsocketEvent):
    item = {
        'PK': 'Connection',
        'SK': event.connection_id,
    }
    dynamodb_table.put_item(Item=item)

# Pure lambda function
@app.lambda_function(name="CreateApple")
def create_apple_step_function(event,context):
    create_apple()

A serverless application is even more readable using Chalice compared to the Serverless Framework. The configuration code and the source code are gathered in the same file. We understand the cause (the trigger) and the effect (the code) of each lambda at a glance.

It is worth mentioning that, with Chalice, only one lambda is deployed for all declared HTTP routes. As a consequence, all the logs will be gathered in a unique log group in CloudWatch.

CDK integration with Chalice: the missing part of the Serverless Framework

Let’s focus on one of the main reasons why I decided to test Chalice instead of using the Python version of the Serverless Framework. Chalice can be integrated with CDK more easily than with the Serverless Framework. CDK resources can be used in Chalice code with environment variables and lambdas defined on the Chalice side can be referred to on the CDK side.

When building an application with Chalice and CDK, you need a repository infrastructure for the CDK application and a runtime repository for the Chalice part. To deploy your full application, you will need to run the cdk deploy command in your infrastructure repository. Here is the architecture of a simple project:

├── infrastructure
│   ├── app.py
│   ├── cdk.json
│   ├── requirements.txt
│   └── stacks
├── requirements.txt
└── runtime
    ├── app.py
    └── requirements.txt

All the code needed to define the lambdas is written on the Chalice side. Other AWS resources are deployed by directly writing CDK Constructs, which is very good at defining complex resources and infrastructure.

Here is an example of the way we can deploy a step-function which refers to lambda function defined on the Chalice side:

# The get_function is used to refer to the lambda function
create_apple = tasks.LambdaInvoke(self, "CreateApple",
               lambda_function=self.chalice.get_function("CreateApple"))

step_function_definition = choose_wait_time.next(wait_x).next(create_apple)
                                                    .next(success)

state_machine = sfn.StateMachine(self, "CreateAppleMachine",
                definition=step_function_definition)

As you can see, I simply use the get_function to refer to the pure lambda CreateApple defined in the previous section. This way, I create the create_apple step that I can integrate into the pipeline of the step_function_definition.

Conclusion

If you need to switch to Python to develop your next serverless application, do not worry. Chalice is a mature serverless framework with a lot of advantages (configuration with decorators, native CDK integration for non-Lambda resources). The tooling surrounding the serverless Python ecosystem has a great maturity. Walking the path from Typescript to Python will not be an easy task, but the destination is worth it!