How to access VPC and internet resources from Lambda without paying for a NAT Gateway

AWS recommend you don't connect Lambda functions to a VPC unless absolutely necessary. This is solid advice because doing so brings several limitations that a standard function doesn’t suffer from.

One of these limitations is that your Lambda function can no longer access the internet. What many people don’t realise is that communicating with other AWS resources inside your account from your function also requires internet access (think S3, SNS, SQS, etc). This makes many common VPC use cases tricky to implement with Lambda.

Let’s take the example of a scheduled Lambda function that runs a daily report by performing a SQL query against an RDS database. Based on this query result, it then adds messages to an SQS queue for processing. VPC access is required to access the RDS database and internet access is required to post to SQS. How can you do both?

The typically recommended solution is to set up a NAT Gateway which allows your VPC-enabled Lambda running in private subnets to connect to a public subnet that has an internet gateway set up.

Eugh! 😩 The last thing I want to be doing is network configuration. Never mind the extra billing cost that I’ll incur since NAT Gateways are billed both by the hour and per GB of data processed. Pretty far from the serverless way.

Using a VPC proxy Lambda function

An alternative solution I’ve used when faced with this problem is to create what I call a “VPC proxy Lambda function”. Instead of having one Lambda function that does all your work, you have two. The first Lambda function, let’s call it runDailyReport (per our earlier example), is the entrypoint. It's triggered by an event (e.g. a CloudWatch schedule rule) and it's NOT configured to run inside the VPC. Its job is to orchestrate all I/O calls that need to be performed.

The second Lambda function is our VPC proxy, let’s call it dbGetReportResults. It’s configured to run inside the VPC and its sole responsibility is to connect to the RDS cluster, perform a query and return the result. It has no triggers configured.

The key thing here is that the runDailyReport function uses the AWS Lambda API to synchronously invoke the dbGetReportResults function, with InvocationType: "RequestResponse". This means that it will wait for the response to be returned before proceeding. Once it gets the result back, it can then parse it and post jobs to SQS.

Despite executing inside the VPC, the dbGetReportResults function is still accessible to functions outside the VPC because the calling function (runDailyReport) interacts with the AWS Lambda service API, which, like all the other AWS service APIs, is internet facing. It does not need to connect directly to the underlying container inside the VPC where the target function is executed.

Limitations

There are a few limitations to be aware of with this approach.

Firstly, since you’re now using 2 Lambdas instead of 1, you will be paying for the execution time of both. While your VPC proxy function is executing and waiting on the database query to return, your entrypoint function will also still be executing (albeit it will be idle while waiting for the proxy function to return). This is why you might hear people saying that one Lambda function directly invoking another Lambda synchronously is an anti-pattern. I’d usually agree with that, but this use case is a valid exception IMO.

Another minor limitation is that there will be a small additional latency as you need to account for the delay in invoking 2 functions in series (cold or warm start) instead of 1. This should not be an issue if your use case is not user facing.

💌 If you enjoyed this article, you can sign up to my weekly newsletter on building serverless apps in AWS.
Originally published at winterwindsoftware.com.