In a project I started working on some time ago I needed an easy way to do redirects from one domain, example.com to a different domain, example-2.com. Normally I would try and use DNS as far as possible, but when I needed to handle example.com/abc and redirect that to example-2.com/abc there was a problem that needed to be solved. The original example.com (from here on referred to as site A) was hosted as a static website, classic S3 and CloudFront. Example-2.com was on a completely different setup (from here on referred to as site B). I could of course add folders in S3 for /abc with a single index.html file with some redirect javascript, but that presents other problems as CloudFront is an CDN and would not automatically fetch index.html from the /abc path. I also needed an easy way to add and remove mappings between site A and B, redirecting siteA/abc to siteB/xyz.
CloudFront functions and the new KeyValueStore to the rescue!!
Architecture overview
There are two parts to this overall architecture, the actual redirect part in CloudFront and a simple API to add and remove mapping keys, powered by ApiGateway and StepFunctions.
To get into some more details, the call flow when a user is redirected would be.
Let's start to build this solution, but before we start....
KeyValueStore limitations
When starting to work with CloudFront KeyValueStores there are some things you need to understand.
The store has the following limits:
- Total size – 5 MB
- Key size – 512 characters
- Value size – 1024 characters
That means that you need to keep your KeyValueStored trimmed, you can't create a massive amount of key value pairs and you can't store massive amount of data, you can't use it as a database. In a normal use-case these limits should not be a problem.
When working with a KeyValueStore you not only need the ARN of the store, you also need the ETag representing the last version of the store. This is important to remember when adding and removing keys.
It's also important to understand that updates are eventual consistent, it does take a couple of seconds for a change to replicate across edge locations.
Create the store
Before we even start doing anything we need to create a KeyValueStore. Navigate to CloudFront section of the Console. The KeyValueStore is well hidden under Functions.
Just click on the Create KeyValueStore and fill in the form.
Or deploy this CloudFormation Template to do it in an automatic way.
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Creates a serverless url redirect
Parameters:
Environment:
Type: String
Description: Environment type, dev, test, prod
AllowedValues:
- dev
- test
- prod
ApplicationName:
Type: String
Description: The application that owns this setup.
Resources:
RedirectKeyValueStore:
Type: AWS::CloudFront::KeyValueStore
Properties:
Comment: !Sub Key Value Store for the ${ApplicationName} ${Environment}
Name: redirect-urls
We will add more and more resources to this template as we go.
Management API
As said, I need a easy way to create, update, and delete redirect mappings. So we create a API Gateway HTTP Api and integrate that with two StepFunctions to create and remove mappings.
This StepFunction will only have two states, and I use the recently released SDK integration.
The first thing we must do is to call DescribeKeyValueStore, we need to get the Etag so we can modify the KeyValueStore.
In the second state, here PutKey we use the ETag returned by describe and the Key and Value that will be sent as data from our API call.
We can create the StepFunctions from the visual editor in the Console, we need one for delete and one to put. Or we add additional resources to our CloudFormation template and deploy that.
PutKeyStateMachineLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub "${ApplicationName}/putstatemachine"
RetentionInDays: 1
PutKeyStateMachine:
Type: AWS::Serverless::StateMachine
Properties:
DefinitionUri: statemachine/put-key.asl.yaml
Tracing:
Enabled: true
Logging:
Level: ALL
IncludeExecutionData: True
Destinations:
- CloudWatchLogsLogGroup:
LogGroupArn: !GetAtt PutKeyStateMachineLogGroup.Arn
DefinitionSubstitutions:
KvsArn: !GetAtt RedirectKeyValueStore.Arn
Policies:
- Statement:
- Effect: Allow
Action:
- logs:*
Resource: !GetAtt PutKeyStateMachineLogGroup.Arn
- Statement:
- Effect: Allow
Action:
- cloudfront-keyvaluestore:DescribeKeyValueStore
- cloudfront-keyvaluestore:PutKey
Resource: !GetAtt RedirectKeyValueStore.Arn
Type: EXPRESS
DeleteKeyStateMachineLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub "${ApplicationName}/deletestatemachine"
RetentionInDays: 1
DeleteKeyStateMachine:
Type: AWS::Serverless::StateMachine
Properties:
DefinitionUri: statemachine/delete-key.asl.yaml
Tracing:
Enabled: true
Logging:
Level: ALL
IncludeExecutionData: True
Destinations:
- CloudWatchLogsLogGroup:
LogGroupArn: !GetAtt DeleteKeyStateMachineLogGroup.Arn
DefinitionSubstitutions:
KvsArn: !GetAtt RedirectKeyValueStore.Arn
Policies:
- Statement:
- Effect: Allow
Action:
- logs:*
Resource: !GetAtt DeleteKeyStateMachineLogGroup.Arn
- Statement:
- Effect: Allow
Action:
- cloudfront-keyvaluestore:DescribeKeyValueStore
- cloudfront-keyvaluestore:DeleteKey
Resource: !GetAtt RedirectKeyValueStore.Arn
Type: EXPRESS
This is the two ASL definitions used to create the actual StepFuntions.
Comment: Put a new or update an existing key in a KeyValueStore
StartAt: DescribeKeyValueStore
States:
DescribeKeyValueStore:
Type: Task
Parameters:
KvsARN: ${KvsArn}
Resource: arn:aws:states:::aws-sdk:cloudfrontkeyvaluestore:describeKeyValueStore
ResultPath: $.DescribeResult
Next: PutKey
PutKey:
Type: Task
Parameters:
IfMatch.$: $.DescribeResult.ETag
KvsARN.$: $.DescribeResult.KvsARN
Key.$: $.Key
Value.$: $.Value
Resource: arn:aws:states:::aws-sdk:cloudfrontkeyvaluestore:putKey
End: true
Comment: Delete an existing key in a KeyValueStore
StartAt: DescribeKeyValueStore
States:
DescribeKeyValueStore:
Type: Task
Parameters:
KvsARN: ${KvsArn}
Resource: arn:aws:states:::aws-sdk:cloudfrontkeyvaluestore:describeKeyValueStore
ResultPath: $.DescribeResult
Next: DeleteKey
DeleteKey:
Type: Task
End: true
Parameters:
IfMatch.$: $.DescribeResult.ETag
KvsARN.$: $.DescribeResult.KvsARN
Key.$: $.Key
Resource: arn:aws:states:::aws-sdk:cloudfrontkeyvaluestore:deleteKey
But, we need a way to invoke the StepFunctions, so for that we create an API Gateway with a Post method and a Delete method and integrate that with our two StepFunctions.
Keep add resources to the template and deploy it, or create it manually from the console.
HttpApi:
Type: AWS::Serverless::HttpApi
Properties:
DefinitionBody:
Fn::Transform:
Name: AWS::Include
Parameters:
Location: api/api.yaml
HttpApiRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: apigateway.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: ApiDirectInvokeStepFunctions
PolicyDocument:
Version: 2012-10-17
Statement:
Action:
- states:StartSyncExecution
Effect: Allow
Resource:
- !Ref PutKeyStateMachine
- !Ref DeleteKeyStateMachine
The actual API definition is in an Open API specification.
openapi: "3.0.1"
info:
title: "update-cloudfront-key-value-store"
paths:
/keyvalue:
post:
responses:
default:
description: "Default response"
x-amazon-apigateway-integration:
integrationSubtype: "StepFunctions-StartSyncExecution"
credentials:
Fn::GetAtt: [HttpApiRole, Arn]
requestParameters:
Input: "$request.body"
StateMachineArn:
Fn::GetAtt: [PutKeyStateMachine, Arn]
payloadFormatVersion: "1.0"
type: "aws_proxy"
connectionType: "INTERNET"
delete:
responses:
default:
description: "Default response"
x-amazon-apigateway-integration:
integrationSubtype: "StepFunctions-StartSyncExecution"
credentials:
Fn::GetAtt: [HttpApiRole, Arn]
requestParameters:
Input: "$request.body"
StateMachineArn:
Fn::GetAtt: [DeleteKeyStateMachine, Arn]
payloadFormatVersion: "1.0"
type: "aws_proxy"
connectionType: "INTERNET"
x-amazon-apigateway-importexport-version: "1.0"
Now we should have an API and we can test to invoke it from Postman and there should be new values added to the KeyValueStore.
Start by checking the KeyValueStore which should show empty key value pairs.
Calling the API from Postman, we supply the ket value pair in the body of the call.
After a call to the API we should now see the value in the key value store.
With that we have a simple API in place to add new values.
I have not added any form of Authorization on the API, if you build this make sure to either add that or tear down the API just after completing your test.
CloudFront setup
Now we have all the bits and pieces in place to create our CloudFront distribution and CloudFront function. There are a couple of things to understand about Cloudfront functions. There are not as broad support as in Lambda@Edge. To fully understand the limitation I recommend that you read some of my other posts.
Protecting a Static Website with JWT and Lambda@Edge or Migrating from Lambda@Edge to CloudFront Functions there is also a very good post by AWS Introducing CloudFront Functions – Run Your Code at the Edge with Low Latency at Any Scale.
The code for our function is not that complicated and looks something like this. The function split the URL and use that as the key. It will then either redirect to the mapped value from the KeyValueStore or if the key doesn't exists, it will redirect to our base url for Site B.
import cf from 'cloudfront';
const kvsId = '${RedirectKeyValueStore.Id}';
const kvsHandle = cf.kvs(kvsId);
async function handler(event) {
const request = event.request;
const headers = request.headers;
const key = request.uri.split('/')[1]
let base = "${BaseSiteUrl}";
let value = ""; // Default value
try {
value = await kvsHandle.get(key);
} catch (err) {
console.log(`Kvs key lookup failed.`);
}
let newurl = base + value
const response = {
statusCode: 302,
statusDescription: 'Found',
headers:
{ "location": { "value": newurl } }
}
return response;
}
So we add the function to out template. Unfortunately we must have the code inline in the template, it's not pretty but at the same time it forces us to keep our functions short.
RedirectFunction:
Type: AWS::CloudFront::Function
Properties:
AutoPublish: true
FunctionCode: !Sub |
import cf from 'cloudfront';
const kvsId = '${RedirectKeyValueStore.Id}';
const kvsHandle = cf.kvs(kvsId);
async function handler(event) {
const request = event.request;
const headers = request.headers;
const key = request.uri.split('/')[1]
let base = "${BaseSiteUrl}";
let value = ""; // Default value
try {
value = await kvsHandle.get(key);
} catch (err) {
console.log(`Kvs key lookup failed.`);
}
let newurl = base + value
const response = {
statusCode: 302,
statusDescription: 'Found',
headers:
{ "location": { "value": newurl } }
}
return response;
}
FunctionConfig:
Comment: Function for url redirect
KeyValueStoreAssociations:
- KeyValueStoreARN: !GetAtt RedirectKeyValueStore.Arn
Runtime: cloudfront-js-2.0
Name: !Sub ${ApplicationName}-${Environment}-redirect-function
We associate the function with the KeyValueStore in the field KeyValueStoreAssociations. What is important to remember is that a function MUST be associated with the KeyValueStore to be able to read it. A function can also only be associated with ONE KeyValueStore. A KeyValueStore can however be associated with several functions.
Finally, we are at a point where we can add our CloudFront distribution and set the CloudFront function to be invoked by the viewer request. Before we continue we need to look at the integration points.
CloudFront integration points
There are four different integration points for Lambda@Edge and you can only set one Lambda function as target for each. The integration is done on the cache behavior so if you have several cache behaviors you can setup integration with different functions for each of them. So how does each integration point work?
Viewer Request The Lambda function is invoked as soon as CloudFront receives the call from the client. This function will be invoked on every request.
Origin Request The Lambda function is invoked after the cache and before CloudFront calls the origin. The function will only be invoked if there is a cache miss.
Origin Response The Lambda function is invoked after CloudFront receives the response from the origin and before data is stored in the cache.
Viewer Response The Lambda function is invoked before CloudFront forwards the response to the client.
However for CloudFront functions, the only available integration points are viewer request and viewer response. We will use the viewer request integration point.
Create CloudFront distribution
When creating the CloudFront distribution I only create a fake origin which only purpose is to integrate with the CloudFront Function. In our use case the function will always return an redirect and we'll never hit the origin.
CloudFrontDistribution:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Aliases:
- !Ref DomainName
Comment: !Sub Distribution for the ${ApplicationName} ${Environment}
CustomErrorResponses:
- ErrorCode: 403
ResponseCode: 200
ResponsePagePath: "/404.html"
DefaultCacheBehavior:
AllowedMethods:
- "GET"
- "HEAD"
- "OPTIONS"
Compress: False
CachePolicyId: 4135ea2d-6df8-44a3-9df3-4b5a84be39ad #Managed Cache Policy 'CachingDisabled'
FunctionAssociations:
- EventType: viewer-request
FunctionARN: !GetAtt RedirectFunction.FunctionMetadata.FunctionARN
TargetOriginId: function-redirect-origin
ViewerProtocolPolicy: redirect-to-https
DefaultRootObject: index.html
Enabled: True
Origins:
- DomainName: !Sub handle-redirect.${DomainName}
Id: function-redirect-origin
CustomOriginConfig:
OriginProtocolPolicy: match-viewer
PriceClass: PriceClass_100
ViewerCertificate:
AcmCertificateArn: !Ref SSLCertificateArn
SslSupportMethod: sni-only
Tags:
- Key: Application
Value: !Ref ApplicationName
- Key: Name
Value: !Sub ${ApplicationName}-${Environment}
With that part deployed our solution is complete. All access to Site A will now be redirected to Site B. Either we redirect to a mapped url found in the KeyValueStore or we redirect to the top of Site B.
Final Words
In this post I showed a way to use CloudFront Functions with KeyValueStore to do redirects between different domains. The KeyValueStore is a very useful part of CloudFront that can be used for different use-case. I have been using CloudFront Functions for other solutions and I have often needed a simple way to store environment variables. The solution presented in this blog could easily be turned into a solution to create, update, and delete exactly that.
I also want to give a shout out to Elias Brange, a good friend that one week before me presented a different useful solution using a similar setup. Elias creates and deploys his solution using SST giving you a different approach to your IaC. We had some good laughs as we wrote similar blogs without knowing it. Hat of to Elias that managed to publish before me.
Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs
Top comments (0)