Knowing cloud services is one thing. Implementing applications is another thing. You won't see a real production workload (hopefully) that wasn't provisioned without any infrastructure as a code tool.
Application Code IS Infrastructure Code
Before serverless & cloud computing it was very common to distinguish infrastructure from application code. For example, your Java Spring server was application code. The database and docker provisioning code was infrastructure.
Nowadays this is different. Think of the combination of Lambda and SQS. It is hard to say what exactly here is infrastructure and what Application.
What is Infrastructure as Code?
Infrastructure as Code (IaC) describes the practice of provisioning your infrastructure with source code. By having source code, you gain many benefits:
- You can duplicate the same environment in another account or region.
- Every change is documented and versioned (thanks to GIT).
- Infrastructure as Code enables you to follow DevOps best practices, such as automated testing, continuous deployment, and feature flagging. All of these practices are possible.
- You get a reproducible setup. There are no manual checklists or steps to remember. Executing the code is sufficient. There are many more benefits. You cannot use AWS professionally without being able to provision resources via Code.
History of IaC
"If you want to understand today, you must search yesterday."
Pearl S. Buck, American novelist (1892-1973)
There are many IaC tools available, and each has its place. To understand their differences, we need to take a step back and understand where we are coming from. Each step in the timeline had a different philosophy.
Remember that cloud computing has only been evolving for the last two decades, and software development has undergone many changes during that time.
The image below shows some of the main categories of different IaC tools.
We started with a manual approach by clicking in the AWS Console. Nowadays, we have several IaC frameworks available and can use them in our favorite programming language. Let's see the single steps in a bit more detail.
Manual - Clicking in the AWS Console
When AWS was launched it was common to create your resources in the UI. This is also called ClickOps. This was very error-prone. It was common to have documents with screenshots and checklists on how to create the resource.
There was no code, history, or governance. With more complex AWS setups this approach became very bad.
Scripted - Provision AWS Resources with the CLI
AWS has a very powerful CLI. At some point, people started using the CLI to provision resources. For example, you can create a bucket like this:
"aws s3api create-bucket --bucket awsfundamentalstestbucket"
These CLI commands were added to pipelines like Jenkins and that is how infrastructure was provisioned.
While this was the first step of going in the direction of having code it was still suboptimal.
The CLI had no idea which resources were already created, or if you needed to update or create new resources, and error handling was also non-existent.
Declarative - Describe the Infrastructure
The next step was using a declarative method. A declarative approach defines what the final state looks like. We know this approach from SQL. We don't care how it will be done; we only define the final state.
For example, we can define that we need an S3 bucket. How the tool takes care of providing us with that bucket is not of interest.
AWS introduced the service CloudFormation. CloudFormation allows you to provision resources, handle errors, and roll back states. A CloudFormation template is a configuration file in a YAML or JSON format. For example, a file could look like this:
CloudFormation has a state and knows which infrastructure was already provisioned and which needs to be updated.
The engine underneath is still used in most of the frameworks like CDK, Amplify, or SST. It is still to this day a very common approach to use CloudFormation.
If you want to use any of the other tools (except for tools like Terraform) you need to understand the basics of CloudFormation.
Tools in this step are for example:
- CloudFormation
- Terraform
- Serverless Framework
Componentized - Use your Programming Language to Build Abstractions
Componentized frameworks allow you to build reusable abstractions that developers can use. There is one big difference between the other stages. Now developers can build infrastructure in a proper programming language. Languages like Python, TypeScript, or Java are very common for these frameworks. Under the hood, CloudFormation will often be used.
Popular frameworks are:
- CDK
- SST (Based on CDK)
- Pulumi Using a proper programming language is the major benefit here. Developers use these languages, concepts, and IDEs daily. There is no need to learn a new syntax. But it is important to learn the basics of the engine underneath, e.g. CloudFormation.
Creating a bucket can be as simple as that:
You have all the benefits of using a programming language but also the benefits that if you don't understand the concepts (or software engineering) well enough, you can easily over-abstract a lot!
Summary
This post gives you an overview of existing IaC tools, their different stages, and why we have them.
Please remember that the stages don't refer to which tool is better suited than the others. Not every tool in the componentized area is better than in the declarative one.
It depends a lot on the organization, use case, and the team that will be working with it.
Top comments (1)
Great insights