Tips for Customizable CloudFormation Templates

This topic is the subject of my presentation at the AWS User Group Mega Manila Meetup last August 20, 2020.

CloudFormation is a powerful tool to define your infrastructure as code. Its declarative syntax provides a way to describe the state of your infrastructure in a JSON or YAML file. It also has a syntax for creating and configuring (almost) any AWS resources you might need.

The limitations of CloudFormation

The limitations of CF start to show when you try to introduce customizability in your templates. For example, you want to create a template that can provision an ECS service for any of your present and future applications. This kind of requirement will make it necessary for the CF template to be customizable. And customizability means you need:

• inputs: In our ECS service example, inputs is a place where the user can choose whether to enable auto-scaling or not. You may also add options for the user to customize how auto-scaling behaves (i.e., like when and how often the scaling happens)
• logic: making sure that the auto-scaling resources are built when the user says he wants auto-scaling.

To this end, AWS has provided a lot of input and logic features that help us create generalize templates. But as you develop more complex CF templates, you will inevitably find these features lacking.

This post is a high-level walkthrough on how to create generalizable templates.

This post is not introductory. There are many good tutorials on the internet for that. Instead, it's a high-level walkthrough on the techniques you can use to make your templates customizable. If you want to dive deeper into any specific solutions, there are links provided at every turn for you to explore. 🚏

We start by introducing features in CF made for template customization. Then, we progress by looking at solutions outside CF that help us truly customizable templates.

(1) Look what CF templates have to offer

Before trying out the more sophisticated solutions in this post, it's essential to ask yourself: is what I'm trying to do already solved by an existing feature in CF templates?. If the answer is yes, then it's better to stick to native CF template syntax.

• Parameters: Variables that you define during development but whose values are only filled up when the user uses your template.
  • Play around with the different parameter types to help your users customize the settings in your template. You can have strings, select boxes (where you define the choices and choose one from them), comma-delimited strings, and even types that allow your users to specify already existing AWS resources like VPC and subnets.
• Pseudo Parameters - Runtime variables defined by the environment you created your stack in, such as AWS::Region, AWS::StackName, etc.
• Intrinsic Functions: Built-in functions in CF that help you manipulate variables whose values are only available during runtime (i.e Parameters, Intrinsic Functions, Properties of resources to be created in the stack)
  • Several of these functions are about manipulating input produced by user input in the parameters section. The most underused of all is the Fn::Sub. This lets you reference CF parameters as if you were doing a string interpolation in Python: ${AWS::StackName}-${UserInputParameter}
  • There also special parameters for creating VPCs that help you make your template more customizable. Check out Fn::GetAZs and Fn::Cidr
• Conditions: a section of the CF template that helps you create a resource by looking if the parameters provided by the user fulfills a specific condition. This syntax is known as the conditional functions (i.e you can use Fn::If, Fn::Equals, Fn::Or, Fn::And, Fn::Not, etc)
• Outputs: this section define the outputs that you will show your users once the stack has successfully created all the resources in the template.
  • This can also help you export values that you want to import and use in other CF stacks that you'd create.

(2) Divide the work between multiple CF templates

As our infrastructure gets bigger, the single CF template we are using becomes unmaintainable. There will also be a lot of repeated code as we may have to deploy two copies of the same resource. An example would be if we wanted to define two separate ECS clusters with each having their own auto-scaling group of EC2 instances. That's 200 lines of YAML per ECS cluster.

• Nested Stack - allows us to create separate templates for components we are reusing. To create two ECS clusters in an environment, we can create a separate ecs_cluster.yml template. Then, we can add two references to this template in our main CF template.
  • All the templates referenced in the main template is created at the same time
• Cross-Stack References - allows us to create stacks that reference values in a stack that already exist. We have to explicitly "export" values from the source template so that other templates can "import" and use them.
  • This allows separate teams to manage different parts of your infrastructure. For example, you can have the infrastructure team manage the stack containing the VPC, Subnets, IGW, and NAT-GW. Then, the core application team manages the stack with the ECS cluster. Individual sprint teams manage ECS services.

In the example below, the networking stack was created first. Then, the database stack references values in the networking stack via cross-stack references. Meanwhile, the ECS service stack nested the CI/CD stack and was created at the same time.

(3) CloudFormation Macros

As you build more complex and customizable templates, the features in #1 and #2 will not be enough. CF Macros allows you to use your own syntax in CF templates.

CloudFormation Macros are lambda functions whose code scans the CF template for special syntax. When it finds that syntax, it uses Python (or any of the supported languages) to add changes to the template before handing it over to the CloudFormation service for processing. It also has access to the value of the parameters supplied when the stack is created.

It also acts as preprocessors to the template. You can create your own macros, or you can just use the ones already publicly available. To use a Macro, deploy the Lambda function and register it as a CloudFormation macro. After registration, you can add the Macro in the Transform section of your CF template.

The AWS CloudFormation Macros GitHub repository is an excellent place to start in looking for macros that might fit your needs. You can also check out jeshan's own curated list in GitHub.

But here is a curated list of Macros from me that you might find useful:

• PyPlate - Run actual Python code in the template. You can place the Python code on a block inside your CF template. During runtime, the PyPlate Macro will look for these blocks and run the Python code inside. Whatever value is assigned to the output variable will be returned to that specific block.
• Count - Add a "Count" property that allows you to create 2, 3, or more copies of that resource

(4) Use Deployment Frameworks for serverless applications

CloudFormation Macros only show you the tip of the customizable available. For serverless applications, two frameworks stand out in their use of Macros.

AWS SAM - Severless Application Model

AWS SAM gives you a shorthand syntax that helps you get started with serverless applications quickly. The CF Macro behind SAM translates the shorthand syntax into the more verbose CF syntax before passing it back to CF for processing.

The template below can deploy a Lambda function already. This would have taken twice the number of lines in CF. Below the template, you can also add vanilla CF code for your Lambda function's AWS resource dependencies (i.e DynamoDB)

Transform: AWS::Serverless-2016-10-31
Resources:
  LambdaFunctionOne:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: python3.8
      CodeUri: 's3://testBucket/mySourceCode.zip'

SF - Serverless Framework

The Serverless Framework is a deployment framework that provides its own syntax to deploy serverless applications across multiple cloud providers. Like AWS SAM, it translates the serverless template to a CF template. Then, it deploys the template as a CF stack. Its "resource" section act as the part where you can dump CF syntax to provision dependencies that your serverless app might need (i.e. DynamoDB)

service: ecs-codedeploy-bluegreen-custom-resource

provider:
  name: aws
  runtime: python3.7
  iamRoleStatements:
    - Effect: "Allow"
      Action: "codedeploy:*"
      Resource: "*"
    - Effect: "Allow"
      Action: "iam:PassRole"
      Resource: "arn:aws:iam::XXXXXXXXXXXXXXX:role/ecsCodeDeployRole"

functions:
  hello:
    handler: handler.hello

plugins:
  - serverless-python-requirements

resources:
  Resources:
    # AWS RESOURCES DEFINED IN CLOUDFORMATION

It also has a vibrant ecosystem of plugins that help you get started even quicker. In another blog post, I discuss how I was able to use the serverless-python-requirements plugin to manage my Quart app's dependencies via SF (Quart is like a better Python Flask Framework). SF will be the one to download the Python packages my app needs, bundle it along with the code I've created, and upload that to an S3 bucket as a ZIP file. That ZIP file will be the code deployed in Lambda

In one of the applications I developed, I used the plugin serverless-wsgi to deploy my Flask application using a WSGI web server.

(5) Look at the CloudFormation Roadmap

CloudFormation has supported a LOT of AWS services, but it hasn't supported everything. It has taken a jab from Terraform lately because Terraform supports new AWS services much quicker than AWS CloudFormation does.

As your use case with CF becomes more specific, you'd come across feature gaps in CF that Terraform already covers. One of them is blue/green deployment for ECS services. The community has been requesting this feature from AWS since January 2019. They were only able to deliver a limited release last May 2020. What a considerable lead time (and it's not even complete!).

In case I encounter such feature gaps, my go-to site is the AWS CloudFormation Coverage Roadmap. Users from all over the world using CF open up issues here to request the AWS CloudFormation team to (i) create new features they'd find useful and (ii) ask the team to cover features already in AWS API, but not yet in CF. The users here also share their workarounds for the problem.

There are other roadmap GitHub repositories, such as this one for ECS. Sometimes users also place feature requests for CF in the roadmap of the service they want to add in CF.

(6) Custom Resources

One of the most common workaround for CF feature gaps is using CloudFormation Custom Resources. Essentially, we create a Lambda function that defines code in Python (or whatever supported language) that creates the custom resource. So for the blue/green example above, I can create a boto3 script that creates a CodeDeploy Deployment Group. Then, I will deploy that boto3 script in Lambda.

Then, in the CF template, we will refer to that Custom Resource in the "Type" field. In the properties field, we are required to define the "ServiceToken" property that defines the ARN of the Lambda function. The rest of the properties are inputs that the Lambda function can use during runtime to create the Custom Resource.

  MyUseCustomLambda:
    Type: Custom::CodeDeployCustomGroup
    Version: "1.0"
    Properties:
      Name: UseCustomLambda
      ServiceToken: arn:aws:lambda:ap-southeast-1:account_id:function:ecs-codedeploy-bluegreen-macro-dev-hello
      ELBName: ""
      TG1Name: !Sub "${AWS::StackName}-blue-tg"
      TG2Name: !Sub "${AWS::StackName}-green-tg"
      GroupName: !Sub "${AWS::StackName}-deployment-group"
      ClusterName: !Ref EcsClusterName         
      ServiceName: !Sub "${AWS::StackName}-service"
      ListenerArn:
        Fn::ImportValue:
          Fn::Sub: "${AANetworkStack}-LoadBalancerListenerArn"   
      DeploymentStyle: BLUE_GREEN

We can also create other resources with CF Custom Resources. We can, for example, use the GCP API and create a server there and have that resource created alongside the AWS resources provisioned in the server itself.

This Medium post by Chris Hare provides an excellent introduction on Custom Resources.

That's all!

How about you? What are the tips for customizability that you can share with us?

I'm happy to take your comments/feedback on this post. Just comment below, or message me!

Special thanks to Alexandre Debiève for the Unsplash cover image.