Introduction
Over the years, the adoption of serverless applications has increased significantly. This has enabled startups to iterate more quickly by deploying proof of concepts and code without having to handle much of the cloud infrastructure. Additionally, serverless applications can help reduce costs because you only pay for the resources you use, rather than paying a fixed cost for running cloud resources 24/7.
While serverless applications have enabled startups to move fast, in order to implement new features and changes with confidence, another essential component is CI/CD (continuous integration/continuous delivery). This enables development teams to automate certain parts of the deployment process and implement safeguards such as mandating that changes pass unit tests before they can be deployed to the cloud.
Before We Start...
In this tutorial, we'll be using AWS SAM, which is AWS's open-source framework for building serverless applications for our sample application.
For the CI/CD tool, we'll be using GitHub Actions, which makes setting up our CI/CD pipeline simpler especially if we're already using GitHub to host our repositories.
This tutorial will also use my sample AWS SAM application using Node.js and TypeScript which I'll share. It uses git submodules as Lambda layers, and is configured to deploy to 3 environments (develop, staging, production).
So without further ado, let's get right on it!
Setting up our AWS SAM Application.
Here's the sample AWS SAM application using Node.js and TypeScript for the purposes of demonstration.
Feel free to clone/fork the repo and set it up for yourself. Setup instructions are in the README file.
Otherwise you can also review the source code, which also contains the GitHub Actions workflow YAML file.
kshyun28 / aws-sam-template-node-ts
Node.js and Typescript serverless boilerplate using AWS SAM template.
aws-sam-template-node-ts
This project is a template for quickly setting up a boilerplate for new APIs. It contains the source code and supporting files for a serverless application using AWS SAM. It includes the following files and folders.
- src - Code for the application's Lambda function written in TypeScript.
- layers - Shared code and dependencies via Lambda layers.
- samconfig.toml - Config for deployment in AWS.
- template.yaml - A template that defines the application's AWS resources.
- .husky - Contains git hooks such as lint-staged and commitlint.
- .commitlintrc.json - Contains config for commit lints.
- .eslintrc.json - Contains config for eslint.
- .lintstagedrc.json - Contains commands ran when committing staged files.
- .prettierrc.json - Contains config for formatting code style.
- jest.config.ts - Contains config for running tests with Jest.
- tsconfig.json - Contains config for development and compilation with TypeScript.
Development Setup
-
Create a new repository using this template. Refer to the documentation for a more detailedโฆ
NOTE: If you already have your own AWS SAM application written in JavaScript/TypeScript, or you're using another language such as Python, feel free to skip this part and proceed to the GitHub Actions setup.
Setting up GitHub Actions for AWS SAM
Step 1: Configuring AWS IAM Role for GitHub Actions
Before we can configure GitHub Actions to our repo, first we will need an AWS IAM Role to provide the necessary permissions needed for GitHub Actions to successfully deploy our AWS SAM application.
NOTE: Although we can also use an IAM User for this, Oleksii Bebych from Automat-IT has a really nice article about using GitHub Actions with AWS IAM roles suggesting that we should use IAM roles for applications and services as outlined in AWS's security best practices in IAM.
While making this blog post, I also saw a recent article on using IAM roles for GitHub Actions made by David Rowe from AWS.
Add AWS IAM Identity Provider
-
First in the AWS console, go to IAM.
-
Next, go to Identity providers, then click Add provider.
-
Then let's configure the identity provider. Use the following configs below.
- For the Provider type, choose OpenID Connect.
- Then for the Provider URL, use https://token.actions.githubusercontent.com.
- Lastly, for the Audience, use sts.amazonaws.com.
-
Optionally, we can also click Get Thumbprint besides the Provider URL field to verify the certificate of the OIDC provider.
Then once you're done reviewing, click Add provider.
Add AWS IAM Role for GitHub Actions
-
Go to Roles while still in the IAM page in AWS console, then click Create role.
Then, choose Web Identity for the Trusted entity type.
-
Choose the Identity provider and Audience we created earlier, which is token.actions.githubusercontent.com and sts.amazonaws.com respectively.
-
Then, let's choose the permissions needed to deploy our AWS SAM application with GitHub Actions.
If you're using my AWS SAM sample project, then these are the permissions needed.
- AWSCloudFormationFullAccess
- IAMFullAccess
- AmazonS3FullAccess
- AWSLambda_FullAccess
- AmazonAPIGatewayAdministrator
- AmazonDynamoDBFullAccess
NOTE: Your permissions may vary depending on the resources your AWS SAM application uses. For example if you're also using SQS and SNS, then you'll need to add permissions for those, vice versa.
Then click Next.
-
Then add a Role name and Description. The changes should look similar below.
Once you're done reviewing the changes, click Create role.
Now we've created our IAM role for GitHub actions, but we're not quite done yet. We'll also need to configure the trust policy in our newly created IAM role.
We will be updating our policy to restrict our IAM role so that it will only allow deployments triggered from our own GitHub repository and branches.
Configure AWS IAM Role Trust Policy
-
First, go to Roles, then select our newly created IAM role GitHubActionsRole (or whatever role name you chose).
-
Next, go to Trust relationships, then click Edit trust policy.
-
Now your trust policy should look something like this.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "sts:AssumeRoleWithWebIdentity", "Principal": { "Federated": "arn:aws:iam::499202726088:oidc-provider/token.actions.githubusercontent.com" }, "Condition": { "StringEquals": { "token.actions.githubusercontent.com:aud": [ "sts.amazonaws.com" ] } } } ] }
Let's update the trust policy with the following. Use your own IAM role ARN and GitHub repository/branches.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::499202726088:oidc-provider/token.actions.githubusercontent.com" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "token.actions.githubusercontent.com:aud": "sts.amazonaws.com", "token.actions.githubusercontent.com:sub": [ "repo:kshyun28/aws-sam-template-node-ts:ref:refs/heads/develop", "repo:kshyun28/aws-sam-template-node-ts:ref:refs/heads/staging", "repo:kshyun28/aws-sam-template-node-ts:ref:refs/heads/production" ] } } } ] }
After making the changes, click Update policy.
This policy change ensures that our IAM role can only be triggered from repositories and branches that we specify.
Now we've configured our IAM role's trust policy, we will now need the IAM role ARN for configuring GitHub actions next, so let's copy it now.
Step 2: Adding a GitHub Workflow YAML file
In order to configure GitHub Actions in our repository, first we will need to create a workflow by adding a YAML file in our source code.
-
In your GitHub repository, click Actions, then click setup a workflow yourself.
-
Copy the GitHub Actions workflow YAML file contents, like in the example below. Then update with the IAM role ARN we copied earlier.
on: push: branches: - develop - staging - production env: BRANCH_NAME: ${{ github.head_ref || github.ref_name }} AWS_REGION: ap-southeast-1 permissions: id-token: write contents: read jobs: build-deploy: runs-on: ubuntu-latest steps: # Checkout with submodules - uses: actions/checkout@v3 with: submodules: recursive # Setup Node.js, feel free to modify with your specific language - uses: actions/setup-node@v3 # Configure AWS SAM CLI and AWS Credentials - uses: aws-actions/setup-sam@v2 - uses: aws-actions/configure-aws-credentials@v2 with: role-to-assume: arn:aws:iam::499202726088:role/GitHubActionsRole role-session-name: aws-sam-template-node-ts-github-actions aws-region: ${{ env.AWS_REGION }} # sam build - run: sam build # Run unit tests - name: Install npm modules run: npm install - name: Run tests run: yarn test # sam deploy - run: sam deploy --no-fail-on-empty-changeset --stack-name stack-name-${{ env.BRANCH_NAME }} --config-env ${{ env.BRANCH_NAME }} --parameter-overrides Environment=${{ env.BRANCH_NAME }}
NOTE: In this example file, I've configured GitHub actions to trigger on 3 specific branches emulating a Gitflow workflow:
- develop
- staging
- productionFeel free to update the following to fit your use case:
- Branches that can trigger GitHub actions
- Whether you're using git submodules or not
- Unit test commands
- SAM CLI commands (sam deploy
, etc.)Then click Start commit and commit the changes.
Testing GitHub Actions
Now that we've finally configured GitHub Actions to deploy to AWS every time we push changes, we can test it out by pushing any changes to the repository and branch we configured earlier.
Conclusion
To summarize what we did to setup GitHub Actions for our AWS SAM application, we created and configured the following:
- IAM Identity Provider for GitHub Actions
- IAM role for GitHub Actions
- IAM role trust policy allow specific repo/branch
- GitHub Actions Workflow file
If you've made it this far, hopefully you were able to follow along and have now learned to setup GitHub Actions as our CI/CD tool for our AWS SAM applications.
There are also other options for other CI/CD tools, most notably AWS CodePipeline + CodeBuild. Paul Swail made a really nice article on why he switched from AWS CodePipeline to GitHub Actions, that's why I leaned towards learning GitHub Actions first.
If you have any feedback, feel free to comment. As this is my first time writing a blog post, there's probably some rough edges here and there.
I'm also available through my email at jasper.d.gabriel@gmail.com, Twitter, LinkedIn, and GitHub.
Thank you for reading and cheers!
References
If you want to learn more, here are some resources that helped me in learning and setting up GitHub Actions for AWS SAM applications.
Top comments (3)
Hey, Jasper.
Great job on your first post. Some great content and references to other complementory articles. I was curious about the way you've configured three separate branches for dev, staging, and production. By the looks of the configuration you're always going to be deploying to the same AWS accout. Do you work in a way where dev, staging, and prod are all in the same AWS account? If so, how are you finding it?
We use separate AWS accounts for each environment and this throws up added complexity to our GitHub Actions Workflows. We have to make use of the upload artifacts workflow and then use different workflows for deploying into said aws accounts.
Hello Luke,
Glad you found the post helpful and my apologies, saw this just now.
Yes the sample project deploys all environments on the same AWS account, though I have to admit it's not ideal.
We've had the same AWS account setup at my previous project, but we quickly found issues and moved away from it.
Instead we also used separate AWS accounts for each environment just like your team's current setup. Dev and staging are in 1 AWS account, then production is in a separate AWS account (for numerous reasons like security, billing, etc.).
Hey, Jasper.
No problem at all! It's interesting that you responded to this message actually as I've recently moved over to a SAM deployment model for the same project that led me to your post.
In the end I used the
sam pipeline
command to create a Github Actions workflow that deploys to a test aws account and then prod aws account. I've seen better CI/CD pipeline implementation but it gets the job done!All the best,
Luke.