This topic is the subject of my presentation at the AWS User Group Mega Manila Meetup last August 20, 2020. The customizability part of my presentation is covered in this post
CloudFormation templates are the least likely of all your code to change once it has been deployed to production. For most small to medium-sized teams, one AWS-savvy person in the group will create the CF template, and it will barely change throughout its lifetime. Since just one person is working on the CF template, there's little incentive to make the template user-friendly. This goes to bite the team when the AWS engineer leaves the company, and those left behind have to update the template in an emergency.
Design CF templates for technical people
The key to addressing this issue is taking the time to improve the user experience. It's probably true that it's a fellow developer who will make changes to the template. We don't have to simplify the template to be appreciated by the common man. We just have to make sure that someone technical with a slight idea of how AWS works can understand the template enough to make changes to it.
From my experience of making CF templates for clients, they usually are somewhat versed in CloudFormation. But of course, they'd want the CF template to be easy to use and maintain. Based on their change requests, I identified three things they usually look for in a template:
- 📚 Readable - can the user know what the CF template is doing just by reading it?
- 🎛 Customizable - how much can the user change with the stack without changing the template itself?
- 🕹 Ease of use - how easy is your template to use?
In this post, I'll present six tricks I do to make the CF template more user-centric and make my clients happy.
(1) Divide your parameters into sections
When you want your templates to be customizable, you'll inevitably end up with way too many parameters. To give you an idea, for the ECS service CF template that I created, I had 29 parameters. Unfortunately for CloudFormation, we cannot add HTML/CSS stylings to divide the 29 parameters into their respective sections: It's just presented as one alphabetically-ordered long list of parameters. This can be overwhelming for most users trying to make use of your template for the first time.
SOLUTION: I prepend the parameter's name based on their section. For the ECS Service template, I have three features (i.e AutoScaling, ServiceDiscovery, ALBConnection), each requiring their own parameters. So I grouped the parameters by prepending the section to the name of each parameter so that when it shows up on CloudFormation, parameters within the same section will be lumped together.
(2) Give your users a choice whether to enable a feature or not
Often, the user will want a choice whether to use a feature in your template or not. A way to do this is by creating multiple templates: "ecs-service-with-auto-scaling", "ecs-service-no-auto-scaling". But this means your code is repeated across templates. This arrangement would be tough to maintain as the number of feature combinations increases.
SOLUTION
- You can have "switch" parameters where you give the user an option to use the feature or not. I usually add "AA" after the section's prefix to make sure the "switch" parameters always end up on top of the section. For the ECS example above, this could mean the switch parameter's name would be: "AutoScalingAASwitch"
- If the feature is not going to be used, the rest of the settings should not be filled up anymore. To add a signifier for this, I prefix the parameter's description with the label "REQUIRED FOR THE SPECIFIC FEATURE TO WORK."
- In the template itself, I add conditions on whether to create a specific service or not. In the template snippet below, the
ECSInstanceProfile
resource is only created ifEc2InstanceProfileAASwitch
= "Yes, add an instance profile".- The
IamInstanceProfile
property of the Ec2Instance still references the ECSInstanceProfile even if it is not created. To remedy this, we added an if statement!If [AddInstanceProfile, !Ref ECSInstanceProfile, !Ref "AWS::NoValue"]
to point the propertyIamInstanceProfile
to NoValue if theECSInstanceProfile
was not created.
- The
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
Ec2InstanceProfileAASwitch:
Description: UNIVERSALLY REQUIRED - Will your Ec2 have an instance profile?
Default: No, it will not have an instance profile
Type: String
AllowedValues:
- Yes, add an instance profile
- No, it will not have an instance profile
Conditions:
AddInstanceProfile: !Equals [!Ref Ec2InstanceProfileAASwitch, "Yes, add an instance profile"]
NoInstanceProfile: !Equals [!Ref Ec2InstanceProfileAASwitch, "No, it will not have an instance profile"]
Resources:
ECSInstanceProfile:
Type: AWS::IAM::InstanceProfile
Condition: AddInstanceProfile
DependsOn:
- ECSRole
Properties:
Path: /
Roles:
- !Ref ECSRole
Ec2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-1234567891011
KeyName:
Ref: BastionHostKeyName
InstanceType:
Ref: BastionHostInstanceType
IamInstanceProfile: !If [AddInstanceProfile, !Ref ECSInstanceProfile, !Ref "AWS::NoValue"]
ECSRole:
...
(3) Use CF Macros to reduce the number of parameters
As you build more customizable templates, you will stumble upon CloudFormation's limitations. One of them is that you have to create dozens of parameters when you want your templates to be customizable. But what if there's a way to group similar parameters?
Parameters:
SecurityGroupIngressRuleString:
Description: REQUIRED - Required for the Security Group of the ECS Service. Make sure to allow access to the assigned SSH Port.
Type: CommaDelimitedList
MinLength: '9'
Default: 80:80:0.0.0.0/0,443:443:sg-1234567890
Resources:
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId:
Ref: VpcId
GroupDescription: Security Group
SecurityGroupIngress: |
#!PyPlate
output = []
for ip_combo in params['SecurityGroupIngressRuleString']:
from_port, to_port, cidr_ip = ip_combo.split(':')
if 'sg' in cidr_ip:
output.append({'IpProtocol': 'tcp', 'FromPort': from_port, 'ToPort': to_port, 'SourceSecurityGroupId': cidr_ip})
else:
output.append({'IpProtocol': 'tcp', 'FromPort': from_port, 'ToPort': to_port, 'CidrIp': cidr_ip})
In the code snippet above, we have the SecurityGroupIngressRuleString
parameter. It is a comma-separated list that specifies the ingress rules for the security group we are creating. Each element of the string contain one ingress rule with the format fromRange:toRange:Destination
:
- The "from range" and "to range" are the range of ports we want to open to the destination.
- The "destination" can either be an IP CIDR (12.33.44.55/32, for example) or a security group ID.
Hence, the statement1000:2000:sg-1234567890
means that we are allowing all resources associated with the sg-1234567890
security group to communicate with the resources associated with this security group via any ports between 1000 and 2000. The user can add as many ingress rules as he likes inside that one parameter.
How does it all work?
We first register the Pyplate Macro by uploading this CF template. This template creates a Lambda function.
When we upload a CF template, CloudFormation looks at the transforms section. Since we added PyPlate in our template snippet above, it finds the PyPlate CloudFormation Macro that we just registered. It triggers the Lambda function and passes the template and our runtime parameters to that function. PyPlate looks at our template line by line and when it sees the "#!PyPlate" string, it runs the Python code. Whatever the variable "output" contains becomes the output for that block of the CF template.
Taking a look at the Python section of the code snippet, we loop through each ingress rule in the params['SecurityGroupIngressRuleString']
list, create a dictionary, and append that to the output variable.
Take it further
You can also create a key-pair style parameter (i.e. "MinCount=1,MaxCount=5"). You can then create the necessary Python code so that MinCount and MaxCount property of a resource will use the respective value in your parameter. With this, you needed to have one parameter instead of 2. (Look at the image at #4 for an illustration).
(4) Create sensible default parameters
Even with all your best effort to reduce the number of parameters the user has to input, sometimes it will not be enough. An alternative is to provide standard defaults to these parameters. The user doesn't have to change these defaults if they aren't particularly sensitive about how the feature works. Maybe they just want it to work out of the box?
An example would be the Auto Scaling feature of the ECS Service template I'm building. Imagine if I left this part blank. The user will have to decide each parameter by himself. By having a default, the user can make the Auto Scaling feature work out-of-the-box. He can return to this later if he wants to finetune the Auto Scaling behavior based on his needs.
(5) Use CF Cross-Stack References to reference values from another stack
Creating sensible defaults still leaves some parameters to be filled up. Parameters like ALB_Listener_ARN, ECS Cluster Name, VPC ID, Subnet IDs need to be filled up with the correct value.
One approach is to use Cross-Stack references. For example, we have the template of the "Networking Stack" below. In its output section, we see that we exported some values. These values are going to be available for other templates to use once this stack has been deployed. Notice how we prepended the AWS Stack Name of this template to make sure the variables are unique should we decide to deploy multiple stacks from this template.
AWSTemplateFormatVersion: '2010-09-09'
Description: 'AWS CloudFormation Sample Template from AWS'
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
EnableDnsSupport: 'true'
EnableDnsHostnames: 'true'
CidrBlock: 10.0.0.0/16
PublicSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: VPC
CidrBlock: 10.0.0.0/24
InternetGateway:
Type: AWS::EC2::InternetGateway
Outputs:
VPCId:
Description: VPC ID
Value:
Ref: VPC
Export:
Name:
Fn::Sub: "${AWS::StackName}-VPCID"
PublicSubnet:
Description: The subnet ID to use for public web servers
Value:
Ref: PublicSubnet
Export:
Name:
Fn::Sub: "${AWS::StackName}-SubnetID"
In a second CF template, we are deploying an EC2 instance. To deploy the instance, we need some values from the Network Stack (i.e the VPC and the subnet). Instead of asking for these parameters as VPC-id and Subnet ID, we ask the name of the CF stack that created the VPC and subnet resources. Then, in the EC2 instance section, we just reference the name of the variable we exported in the "Network" Stack. We append the name of the stack to form the variable: "${NetworkStackName}-SubnetID".
AWSTemplateFormatVersion: 2010-09-09
Description: AWS CF Cross-Stack Reference Sample Template
Parameters:
NetworkStackName:
Description: Stack Name of the Network Stack
stack.
Type: String
Default: SampleNetworkCrossStack
Resources:
WebServerInstance:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.micro
ImageId: ami-05a66c68
NetworkInterfaces:
- GroupSet:
- Fn::ImportValue:
Fn::Sub: "${NetworkStackName}-SecurityGroupID"
AssociatePublicIpAddress: 'true'
DeviceIndex: '0'
DeleteOnTermination: 'true'
SubnetId:
Fn::ImportValue:
Fn::Sub: "${NetworkStackName}-SubnetID"
(6) Use a local preprocessor to create profiles
An alternative to cross-stack references is using a local preprocessor to prefill default values. I did this by creating a simple Python script in my local and running that script before uploading the template.
import cfn_flip.yaml_dumper
import yaml
from cfn_tools import load_yaml
# (1) read the template
with open("raw_template.yml") as f:
raw = f.read()
cf_template = load_yaml(raw)
# (2) read the profile definition
with open("blogsite-profile.yml") as f:
raw = f.read()
profile_definition = load_yaml(raw)
# (3) merge the values of the profile definition as the default value for its respective parameter in the template
for parameter_key in profile_definition:
parameter_value = profile_definition[parameter_key]
cf_template['Parameters'][parameter_key]['Default'] = parameter_value
# (4) write the new template to a new YAML file
with open("output_template.yml", 'w') as f:
dumper = cfn_flip.yaml_dumper.get_dumper(clean_up=False, long_form=False)
raw = yaml.dump(
cf_template,
Dumper=dumper,
default_flow_style=False,
allow_unicode=True
)
f.write(raw)
First, we have the CF template we use to create the resources. We also have another YAML file that contains the values we want to insert in our template's parameters (let's call this profile_definition). In the 4th code block in our Python script, we go through each parameter in the profile_definition, find the equivalent parameter for it in the cloudformation template, and set the parameter's default value equal to the value of the parameter in the profile_definition.
The process looks like this:
- You can create profile_definitions for different environments / applications you might have
- We fill in the default value of the parameter to provide a standard for the application. But the user can still change these values should he need to.
That's all!
How about you? What are the tricks you can share to make your AWS CloudFormation templates more user-centric?
I'm happy to take your comments/feedback on this post. Just comment below, or message me!
Special thanks to Daniel McCullough for the background image
Top comments (0)