AWS Managed Prefix lists are a really powerful way of abstracting the details of CIDR Blocks into something meaningful for the humble cloud engineer.
Much like DNS does for IP addresses to hostnames, we can create a managed prefix for any service that has several CIDR blocks associated with it.
I've talked about this before here, but with more and more stuff needing to talk to services hosted on the internet and with our implementation of AWS Network Firewall, it is time for a re-visit.
The challenge
Most people are familiar with the AWS Public IP Address Ranges source ip-ranges.json, doco linked here, released in 2014. Can you believe it!
It provides all the Public IP ranges for all services across all regions. A very handy thing to have, and with it being hosted in a JSON file, we can automatically parse it with any automation.
But why don't AWS do this for us? I can't answer that today; however, a good example of such automation exists in the AWS samples GitHub here.
The AWS Samples repo does all the heavy lifting with Lambda and creates the following:-
But I'm feeling hacky and need some Organization-level sharing, Lambda PowerTools, AppConfig and deployment through SAM CLI.
Let's get into it!
Some Enhancements
Lambda PowerTools
As the homepage displays, Lambda PowerTools is a suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, idempotency, batching, and more.
One of the reasons for implementing this is to make the Cloudwatch logs nicer to parse. It also removes a few lines of code in Lambda without requiring major logging structure changes.
As I've rebuilt this into SAM Cli, this is just an addition to the requirements.txt file to include aws-lambda-powertools.
AppConfig
If you look at the original AWS example, it requires a services.json to be defined to configure what regions and services to create the WAF IPsets and VPC Prefixes. It will look something like the following:-
{
"Services": [
{
"Name": "CODEBUILD",
"Regions": [
"ap-southeast-2"
],
"PrefixList": {
"Enable": true,
"Summarize": true
},
"WafIPSet": {
"Enable": true,
"Summarize": true,
"Scopes": [
"REGIONAL"
]
}
}
]
}
This is in JSON format, so perfect for AppConfig.
As we have configured Lambda PowerTools, AppConfig is easily added as a Lambda Layer.
I've made several updates to the template.yaml file to define our AppConfig hosted configuration and deployment methods, adding environment variables and our Lambda layer.
Parameters:
AppConfigAppName:
Type: String
Description: AppConfig Application Name
Default: aws-ip-ranges
AppConfigAppEnvironmentName:
Type: String
Description: AppConfig Application Environment Name
Default: dev
AppConfigName:
Type: String
Description: AppConfig Name
Default: services
AppConfigLayerArn:
Type: String
Description: Retrieve AWS AppConfig Lambda extension arn from `https://docs.aws.amazon.com/appconfig/latest/userguide/appconfig-integration-lambda-extensions-versions.html#appconfig-integration-lambda-extensions-enabling-x86-64`
Default: arn:aws:lambda:ap-southeast-2:080788657173:layer:AWS-AppConfig-Extension:91
AwsOrgArn:
Type: String
Description: The ARN of the AWS Organization used to share Prefix Lists
Default: notset
Resources:
SAMConfigApplication:
Type: AWS::AppConfig::Application
Properties:
Name: !Ref AppConfigAppName
Environment:
Type: AWS::AppConfig::Environment
Properties:
Name: !Ref AppConfigAppEnvironmentName
ApplicationId: !Ref SAMConfigApplication
SAMConfigConfigurationProfile:
Type: AWS::AppConfig::ConfigurationProfile
Properties:
ApplicationId: !Ref SAMConfigApplication
Name: !Ref AppConfigName
Type: 'AWS.Freeform'
LocationUri: 'hosted'
SAMConfigDeploymentStrategy:
Type: AWS::AppConfig::DeploymentStrategy
Properties:
Name: "SAMConfigDeploymentStrategy"
Description: "A deployment strategy to deploy the config immediately"
DeploymentDurationInMinutes: 0
FinalBakeTimeInMinutes: 0
GrowthFactor: 100
GrowthType: LINEAR
ReplicateTo: NONE
BasicHostedConfigurationVersion:
Type: AWS::AppConfig::HostedConfigurationVersion
Properties:
ApplicationId: !Ref SAMConfigApplication
ConfigurationProfileId: !Ref SAMConfigConfigurationProfile
Description: 'AWS Service configuration for update-aws-ip-ranges'
ContentType: 'application/json'
Content: |
{
"Services": [
{
"Name": "CODEBUILD",
"Regions": [
"ap-southeast-2"
],
"PrefixList": {
"Enable": true,
"Summarize": true
},
"WafIPSet": {
"Enable": true,
"Summarize": true,
"Scopes": [
"REGIONAL"
]
}
}
]
}
AppConfigDeployment:
Type: AWS::AppConfig::Deployment
Properties:
ApplicationId: !Ref SAMConfigApplication
ConfigurationProfileId: !Ref SAMConfigConfigurationProfile
ConfigurationVersion: !Ref BasicHostedConfigurationVersion
DeploymentStrategyId: !Ref SAMConfigDeploymentStrategy
EnvironmentId: !Ref Environment
LambdaUpdateIPRanges:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: src/
Handler: app.lambda_handler
Runtime: python3.9
Architectures:
- x86_64
Environment:
Variables:
APP_CONFIG_APP_NAME: !Ref AppConfigAppName
APP_CONFIG_APP_ENV_NAME: !Ref AppConfigAppEnvironmentName
APP_CONFIG_NAME: !Ref AppConfigName
AWS_ORG_ARN: !Ref AwsOrgArn
LOG_LEVEL: INFO
Layers:
- !Ref AppConfigLayerArn
In order to pull in the config, we add the following function and a few adjustments in the handler code.
def get_service_config():
try:
appconfig = f"http://localhost:2772/applications/{APP_CONFIG_APP_NAME}/environments/{APP_CONFIG_APP_ENV_NAME}/configurations/{APP_CONFIG_NAME}"
with request.urlopen(appconfig) as response: # nosec B310
config = response.read()
return config
except Exception as error:
logger.error("Error retrieving AppConfig configuration. Exiting")
logger.exception(error)
raise error
Organization-level sharing
To enable all the users in our organization to benefit from our automation, we need to share the VPC Prefixes via Resource Access Manager (RAM).
We define an additional function to create the RAM share on creating the VPC Prefix List.
def create_prefix_ram(client: Any, prefix_list_name: str, prefix_list_arn: str) -> None:
"""Create the VPC Prefix List RAM Share"""
logger.info("create_prefix_ram start")
logger.debug(f"Parameter client: {client}")
logger.debug(f"Parameter prefix_list_name: {prefix_list_name}")
logger.debug(f"Parameter prefix_list_arn: {prefix_list_arn}")
logger.info(f'Creating RAM Share "{prefix_list_name}" with Arn "{prefix_list_arn}"')
response = client.create_resource_share(
name=prefix_list_name,
resourceArns=[prefix_list_arn],
principals=[AWS_ORG_ARN],
allowExternalPrincipals=False,
tags=[
{"key": "Name", "value": prefix_list_name},
{"key": "ManagedBy", "value": MANAGED_BY},
{"key": "CreatedAt", "value": datetime.now(timezone.utc).isoformat()},
{"key": "UpdatedAt", "value": "Not yet"},
],
)
logger.info(f'Created VPC Prefix List RAM Share"{prefix_list_name}"')
logger.debug(f"Response: {response}")
logger.debug("Function return: None")
logger.info("create_prefix_ram end")
NOTE: All the code is available at https://github.com/sjramblings/update-aws-ip-ranges
The End Result
With our updates, this is now our enhanced Lambda workflow
- Update to
arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChangedwill trigger the function - Lambda will execute under the IAM Role, retrieve its service config from AppConfig and write output to Cloudwatch
- For enabled services, it will create/update the VPC Prefixes & WAF IPSets
- VPC Prefixes will be shared with the Organization ID via RAM
A few screenshot of the results:-
Summary
The wonderful contributors to AWS-Samples had done a great job in getting us started. However, I needed to take it a bit further, adding the VPC Prefix Sharing and a few other tweaks.
As noted above, all code is available at https://github.com/sjramblings/update-aws-ip-ranges. I will also raise Pull Requests to backport some of this into the AWS Samples repo.
I hope this helps someone else!
Cheers
Top comments (0)