Part 4a: Terraform IaC, GitLab CI and automated testing

Included in this chapter...

• Technology Used
• Objective
• Preparation
• Execution
• Architecture Diagram
• Summary

Technology Used

• AWS
• Cypress
• GitLab CI
• Terraform
• Visual Studio Code / bash / environment variables…

Objective

At the end of this step, the frontend website will look no different however the backend configuration will be completely reformed. The key differences will be:

1. The entire AWS configuration will have been set up as Infrastructure as Code using Terraform, all deployable with only a few clicks.
2. All configurations will live in a GitLab repository dedicated to this project.
3. If any changes have been made to the index.html file, when pushed to the GitLab repo, this will trigger a pipeline that will run tests to ensure that updated content meets predefined rules.

Preparation

I took a comprehensive deep-dive course on Terraform where I learnt how to use it from the ground up. Afterwards, with a good understanding of how it works, I began to apply this knowledge to create the Terraform configuration in parallel with the AWS console created infrastructure.

The order of my creation was:

1. DynamoDB
2. IAM
3. Lambda
4. API Gateway
5. S3
6. CloudFront
7. Certificate Manager / CloudFlare

I also took courses in GitLab and Cypress so I was later able to configure a GitLab repo where I would store all the code as well as setting up an automation pipeline using GitLab CI and Cypress to ensure the index.html file meets certain testing criteria before uploading to the S3 bucket.

I have separated this blog into two posts, this one you are reading, with an overview of how I solved this part of the Cloud Resume Challenge and a second post (Part 4b: Problem solving and debugging) detailing several issues I had and how I was able to overcome each of them.

Execution

Within Terraform I decided to set up the configuration using a modular structure as opposed to adding everything to the main.tf file. The project file structure I used is shown below. I set up the configuration this way as it allows for greater flexibility and readability.

Project file structure:

├── cypress
│   └── integration
│       └── index-resume-test-tf.cy.js
├── terraform_config
│   ├── acm_module
│   │   └── acm.tf
│   ├── api_gateway_module
│   │   └── api_gateway.tf
│   ├── cloudfront_module
│   │   └── cloudfront.tf
│   ├── dynamodb_module
│   │   └── dynamodb.tf
│   ├── iam_module
│   │   └── iam_role.tf
│   ├── lambda_module
│   │   ├── lambda_code.py
│   │   ├── lambda_code.zip
│   │   └── lambda.tf
│   ├── s3_module
│   │   └── s3.tf
│   └── main.tf
├── website_files
├── .gitlab-ci.yml
└── cypress.config.js

During the development of the Terraform IaC stage I have become comfortable using the Terraform Registry for creating each service to match my AWS console created version.

This part of the challenge involved a fairly solid chunk of work and along the way I encountered several issues that slowed down my progress but with a considerable amount of testing and persistence to see it through I was able to overcome each challenge.

This has been by far the most complicated part of the Challenge, the learning level was steep however I have learnt a lot about using and configuring Terraform, setting up automation pipelines and inflight testing using GitLab and Cypress and I have successfully built an end-to-end application using these and other important cloud technologies.

The end result can be seen here 🙂

www.simon-green.uk

Architecture Diagram

The diagram shown below illustrates the final overall end-to-end architecture that I created for this section and the project.

Summary

I have completed the Cloud Resume Challenge and in doing so I have gained a wealth of knowledge and hands-on experience in developing a cloud based application on AWS. I have encountered many issues but I have solved each in a logical and methodical manner to overcome each challenge.

It has been a fulfilling learning experience which I aim to continue building on through additional projects and ultimately a career move.

Don't forget to check out my next post Part 4b: Problem solving and debugging to read about a selection of the issues I faced and how I was able to resolve them.