A/B Testing, Blue/Green deployments, Canary releases. Different, but still so much in common. They all have different purpose but are using basically using the same technical solution under the hood. We do these kind of tests for several reasons. One is to do a A/B test and determine what version our users like the best. Sure it is possible to do A/B testing on the client side, but personally I find it easier to do server side.
Blue / Green and Canary deployments are done to make sure a new version of the application work as we expect and give us an easy way to roll back to previous version in case of a problem. All of these are important practices in the DevOps culture.
Many services from AWS offer this solution out of the box, one service that doesn't is CloudFront. Luckily CloudFront has the possibility to run Lambda@Edge which we can use to solve this.
All source code for this setup is found on GitHub
Introducing Lambda@Edge
Lambda can run in four different locations in the request flow.
Viewer request - Is run when CloudFront receives a request from a viewer.
Origin request - Is run before CloudFront forwards a request to the origin.
Origin response - Is run when CloudFront receives a response from the origin.
Viewer response - Is run before CloudFront returns the response to the viewer.
There are several limitations when it comes to Lambda@Edge, it is only possible to create functions in Python and Node.js. Viewer request and response functions can only allocate 128mb of memory and only run for 3 seconds. You can't use environment variables, you can't use the latest version alias, only fixed versions are supported. Logs are published to the edge region that you access and not to us-east-1 region, even if functions must be deployed to that region. Be sure to read the documentation before you start working with Lambda@edge.
Solution Overview
In this solution we are going to use two different Lambda functions for viewer request and origin response hooks, we will store configuration in Parameter Store, and S3 is our origin.
In the viewer request hook we will check for a special cookie, if the cookie is not set we will fetch a random version. In the origin response we set the set-cookie header to store the version we are using.
By setting different values in Parameter Store we can control the weight of each version and we can also reset and have clients receive new random version.
Viewer Request
Let's start with the Lambda function that will react to the Viewer Request Event. This Lambda function is responsible for checking for our cookie X-Version-Name if an value is set the function will update the request path based on the cookie value. If there is no value set it will roll the dice and get a random version and update the request path. This function will also match the value in cookie X-Version-Reset towards a value in Parameter Store and if the value is different it will ignore any set value in X-Version-Name. If there is no value in X-Version-Name or if the value is ignored the function throws the dice and picks a version at random. The weight, for the different versions, are controlled by a value in Parameter Store. Finally the function pass the cookie to the next step in the call chain.
The Viewer Request code
Full version is available in GitHub.
def lambda_handler(event, context):
request = event['Records'][0]['cf']['request']
headers = request['headers']
cookie_version_blue = 'X-Version-Name=Blue'
cookie_version_green = 'X-Version-Name=Green'
path_blue = '/Blue'
path_green = '/Green'
uri = ''
if request['uri'].endswith('/'):
request['uri'] = request['uri'] + 'index.html'
if 'cookie' not in request['headers']:
request['headers']['cookie'] = []
# Reset weights, ignore already set cookie
reset_weight, reset_cookie = do_weight_reset(headers)
if not reset_weight:
for cookie in headers.get('cookie', []):
if cookie_version_blue in cookie['value']:
uri = path_blue + request['uri']
break
elif cookie_version_green in cookie['value']:
uri = path_green + request['uri']
break
request['headers']['cookie'].append(
{'key': 'Cookie', 'value': reset_cookie})
if not uri:
weight = int(load_parameter('Weight'))
cookie_value = ''
if random.random() < float(weight / 100.0):
uri = path_blue + request['uri']
cookie_value = cookie_version_blue
else:
uri = path_green + request['uri']
cookie_value = cookie_version_green
request['headers']['cookie'].append(
{'key': 'Cookie', 'value': cookie_value})
request['uri'] = uri
return request
Viewer Response
The Viewer Response function basically has one task, and that is to pass set-cookie header to the client. This is needed so the client set the cookies X-Version-Name and X-Version-Reset so the client send them in the next request. This is important so the client doesn't jump between versions. The function has a small but very important job to do.
The Viewer Response code
Full version is available in GitHub.
def lambda_handler(event, context):
response = event['Records'][0]['cf']['response']
request = event['Records'][0]['cf']['request']
# Persist cookie, set the set-cookie header
if 'set-cookie' not in response['headers']:
response['headers']['set-cookie'] = []
request_headers = request['headers']
cookie_version_blue = 'X-Version-Name=Blue'
cookie_version_green = 'X-Version-Name=Green'
cookie_reset = 'X-Version-Reset'
for cookie in request_headers.get('cookie', []):
if cookie_version_blue in cookie['value']:
response['headers']['set-cookie'].append(
{'key': 'set-cookie', 'value': cookie_version_blue})
elif cookie_version_green in cookie['value']:
response['headers']['set-cookie'].append(
{'key': 'set-cookie', 'value': cookie_version_green})
elif cookie_reset in cookie['value']:
response['headers']['set-cookie'].append(
{'key': 'set-cookie', 'value': cookie['value']})
return response
Deploying the functions
The Lambda functions need to be deployed in us-east-1 region, since that is the region Lambda@Edge originates from. We must also use a fixed version and can't use the latest alias. As normal AWS SAM is used to define and deploy Lambda Functions.
The SAM Template
Full version is available in GitHub.
ViewerRequestFunction:
Type: AWS::Serverless::Function
Properties:
AutoPublishAlias: "true"
Runtime: python3.7
MemorySize: 128
Timeout: 3
CodeUri: ./viewer-request
Handler: handler.lambda_handler
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- edgelambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- SSMParameterReadPolicy:
ParameterName: !Sub ${SsmConfigPath}/*
- Version: "2012-10-17"
Statement:
Action:
- lambda:GetFunction
Effect: Allow
Resource: "*"
ViewerResponseFunction:
Type: AWS::Serverless::Function
Properties:
AutoPublishAlias: "true"
Runtime: python3.7
MemorySize: 128
Timeout: 3
CodeUri: ./viewer-response
Handler: handler.lambda_handler
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- edgelambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- Version: "2012-10-17"
Statement:
Action:
- lambda:GetFunction
Effect: Allow
Resource: "*"
CloudFront setup
Finally we need to create the CloudFront distribution and set the Lambda functions for the Viewer Request and Response triggers. Normally CloudFront will not use and pass the headers to the cache. Since our setup is depending on two cookies we must make sure CloudFront pass them along. That is done by adding them to WhitelistedNames section. Wildcards are supported so we just add X-Version-* to that section.
The CloudFormation Template
Full version is available in GitHub.
CloudFrontDistribution:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Comment: !Sub "Distribution for ${ProjectName}"
DefaultCacheBehavior:
AllowedMethods:
- "GET"
- "HEAD"
- "OPTIONS"
Compress: False
DefaultTTL: 0
MaxTTL: 0
MinTTL: 0
ForwardedValues:
QueryString: False
Cookies:
Forward: whitelist
WhitelistedNames:
- "X-Version-*"
LambdaFunctionAssociations:
- !If
- ViewerRequestLambdaArnSet
- EventType: viewer-request
LambdaFunctionARN: !Ref ViewerRequestLambdaArn
- !Ref AWS::NoValue
- !If
- ViewerResponseLambdaArnSet
- EventType: viewer-response
LambdaFunctionARN: !Ref ViewerResponseLambdaArn
- !Ref AWS::NoValue
TargetOriginId: !Sub ${ProjectName}-origin
ViewerProtocolPolicy: redirect-to-https
DefaultRootObject: !Ref DefaultRootObject
Enabled: True
Origins:
- DomainName: !Sub ${StorageBucket}.s3.amazonaws.com
Id: !Sub ${ProjectName}-origin
S3OriginConfig:
OriginAccessIdentity: !Sub origin-access-identity/cloudfront/${OriginAccessIdentity}
PriceClass: PriceClass_100
Tags:
- Key: Name
Value: !Sub ${ProjectName}
Conclusion
Even though CloudFront doesn't support these deployments techniques out of the box Lambda, as so many times before, come to the rescue. The versatility of AWS Lambda is truly a miracle! Download the code and take it for spin!
Happy hacking!
Top comments (0)