Shortcuts for Debugging Apps on AWS Lambda

With Lambda, you'll spend a lot of time fighting timeouts, invocation errors & other bugs. But there are many helpers out there.

I'm a huge fan of automating things and making my developer life as easy as possible. Personally, this especially affects how to handle issues.

If something breaks or goes wrong, I want to immediately know the reasons and not have to fight with CloudWatch's user interface and navigate through endless logs until I find out what's going on.

The Problem Statement

Things break. That's a matter of fact and serverless is not going to change this.
Your Lambda functions can timeout, fail to be invoked, have dependency issues, and much more.

What's important: get to the cause as easily as possible.

The Toolbox

The tools you're working with can be a swiss army or butter knife: it depends on your choices and how you use them.

After working with Lambda for years, I finally settled for three major helpers:

• CloudWatch for Humans - bringing CloudWatch logs to your terminal
• Dashbird.io - a real-time error tracking tool for AWS Lambda
• Alfred - a spotlight alternative

Using those efficiently helped me to reduce my operational burdens immensely. Let me explain how.

CloudWatch for Humans

With Lambda, you'll be using CloudWatch in some way. Due to Lambda's internal structure with running micro containers which all create their own log stream, it's not an easy task to correlate your logs and find what you want. To make it worse, the CloudWatch console interface is not the most intuitive or user-friendly.

That's why I love AWS CloudWatch logs for Humans which brings CloudWatch logs to your terminal.

Browsing & finding your log groups

If you're having a lot of functions and probably environments on the same account, it can be even tricky to find your target log group. With awslogs groups and grep, this will be much easier.

#  listing log groups for eu-central-1
awslogs groups --aws-region=eu-central-1
# show group names containing 'myfunction'
awslogs groups | grep 'myfunction'

Streaming logs to your terminal

As said earlier: it's not fun to browse logs in CloudWatch's console. With awslogs get, we can bring our logs to the terminal.

# displaying the last logs of our target service
awslogs get /aws/lambda/myfunction
# starting a continuous stream of logs
awslogs get /aws/lambda/myfunction --watch
# receiving the last 15 minutes of a log streamawslogs get /aws/lambda/myfunction --start='15m ago'

Filtering logs

Logs can be hard to read, especially if you're logging complex JSON. Additionally, there can be way too many logs. By filtering, we can reduce it to what we're actually looking for.

# get logs with upstream_status 502
awslogs get /aws/lambda/myfunction --filter-pattern='{ $.upstream_status = 502 }'
# only displaying the field logMessage of the JSON logawslogs get /aws/lambda/myfunction --query=logMessage

Dashbird

Serverless promised us a lot fewer operations that we need to handle - which is indeed true. But operating a distributed Lambda-powered architecture still requires the Ops in DevOps. It's possible to do this just with AWS-native tools, but it's cumbersome. Relying on a third-party tool for enhanced observability is much easier.

That's what Dashbird is offering: observability and real-time error tracking for your serverless application running on AWS. It easily integrates with your AWS account via a CloudFormation stack that enables Dashbird to query data via AWS APIs.

It then discovers all resources and helps you to detect not only current but also potential future issues and performance bottlenecks at one glace.

Alfred

This is probably my most used tool of all time. It's "spotlight on steroids" with a lot of great features making your daily developer life easier.

What I like the most: creating my own workflows to automate recurring activities or just to quickly jump to any app.

Example: a small mapper for quickly jumping to my Lambda functions inventory on Dashbird so that I can get to the root of issues easily.

That is just one of many uses cases as the automation possibilities of Alfred are just limitless. There are a lot of official workflows and there's also a huge community building & offering workflows for anything you can think of.