Schedule Access Key Rotation Using CodeCatalyst

To interact with services/resources in Amazon Web Services (AWS), commonly We need security credentials. Security credentials are needed for authentication and authorization. There are two types of security credentials, long term and temporary. Both have its own use cases, but temporary credentials is more recommended.

Despite using temporary security credentials is more recommended, maybe in some use cases We still need or have to use long term credentials such as IAM user access key and secret key. To prevent security hole, We can and encouraged to rotate access key and secret key periodically.

More details about AWS security credentials: [+]

Table of Contents

1. Scenario
2. Prerequisites
3. Configuration Steps
   • Directory Structure
   • Access Key Rotation Script Configurations
   • CodeCatalyst Workflow File Configurations
   • Example of Automate Access Key Rotation in CI/CD CodeCatalyst
4. Summary

Scenario

In this blog, I will create CI/CD pipeline in Amazon CodeCatalyst or in CodeCatalyst also called as workflow, where this workflow will run scheduled automated IAM user access key rotation. First, CodeCatalyst will delete old access key, then it will create new access key. In the process, CodeCatalyst workflow will run shell scripts that has been configured to run aws cli commands that needed for each steps.

Prerequisites

Here are some prerequisites that needed to follow configurations discussed in this blog:

• CodeCatalyst space and project that has been connected with AWS account and source repository. In this blog, I use GitHub as source repository. For configuration reference, please check my other blog “Build and Release Container Image to Amazon Elastic Container Registry (ECR) via Amazon CodeCatalyst”
• IAM user

Configuration Steps

Directory Structure

Below is directory structure used in this blog:

.
├── .codecatalyst/
│   └── workflows/
│       └── rotate-access-key.yml
└── sandbox2/
    └── iam/
        ├── create-access-key.sh
        └── delete-access-key.sh

Access Key Rotation Script Configurations

Configuration started with create shell scripts that will be used for automate access key rotation.

1. Create shell script delete-access-key.sh that will be used to check whether IAM user has access key and if user has, script will delete that access key. In this script, We need to export environment variable value IAM_USER that used to define which IAM user will be checked
   

   #!/usr/bin/env bash
   
   set -euo pipefail
   
   IAM_USER=$IAM_USER
   
   ACCESS_KEY=$(aws iam list-access-keys \
     --user-name $IAM_USER \
     --query 'AccessKeyMetadata[].AccessKeyId' \
     --output text \
     --no-cli-pager)
   
   if [ -z "$ACCESS_KEY" ]; then
     echo "No access key found on user $IAM_USER"
   else  
     echo "Found access key $ACCESS_KEY on user $IAM_USER"
   
     DELETE_KEY=$(aws iam delete-access-key \
       --user-name $IAM_USER \
       --access-key-id $ACCESS_KEY)
   
     echo "Access key $ACCESS_KEY deleted"
   fi
   

2. Create shell script create-access-key.sh that will be used to create/generate new access key. Similar with delete-access-key.sh, We need to export enviroment variable value IAM_USER. Output of access key creation will be saved into file called access-key.csv
   

   #!/usr/bin/env bash
   
   set -euo pipefail
   
   IAM_USER=$IAM_USER
   
   CREATE_KEY=$(aws iam create-access-key \
     --user-name $IAM_USER\
     --output text)
   
   echo "Creating access key for user $IAM_USER"
   echo "$CREATE_KEY" > ./access-key.csv
   echo "Access key created. Please find the key values on access-key.csv"
   
   

CodeCatalyst Workflow File Configurations

After created shell script for automate access key rotation, then We need to define workflow file that will be used to schedule automation of access key rotation.

I configured that workflow in file .codecatalyst/workflows/rotate-access-key.yml. Here is the full content of workflow file:

Name: rotate-access-key
SchemaVersion: "1.0"

Triggers:
  - Type: SCHEDULE
    Expression: "30 12 ? * THU *"
    Branches:
      - master

Actions:
  delete-access-key:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
      Variables:
        - Name: IAM_USER
          Value: "tf-admin"
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: codecatalyst-admin
    Configuration:
      Steps:
      - Run: cd sandbox2/iam
      - Run: ./delete-access-key.sh
    Compute:
      Type: EC2
  create-access-key:
    DependsOn:
      - delete-access-key
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
      Variables:
        - Name: IAM_USER
          Value: "tf-admin"
    Outputs:
      Artifacts:
        - Name: ACCESSKEY
          Files:
            - sandbox2/iam/access-key.csv
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: codecatalyst-admin
    Configuration:
      Steps:
      - Run: cd sandbox2/iam
      - Run: ./create-access-key.sh
    Compute:
      Type: EC2

Next I will explain more details each section inside workflow file.

1. Define workflow name. Then define how workflow run. In this blog, as an example, workflow will be ran as scheduled on Thursday at 12:30 UTC and only will be ran on master branch. Please adjust schedule following cron expression guide that used by CodeCatalyst. CodeCatalyst use six-field syntax and separated with space, more details please check this documentation: CodeCatalyst workflow triggers expression & EventBridge cron expressions reference
   

   Name: rotate-access-key
   SchemaVersion: "1.0"
   
   Triggers:
     - Type: SCHEDULE
       Expression: "30 12 ? * THU *"
       Branches:
         - master
   

2. Define actions. Action will be divided into 2 sections. First section will run shell script delete-access-key.sh in action delete-access-key. As mentioned above, we need to export environment variable value IAM_USER. In this blog, I will use example user called tf-admin. Action will be ran in sandbox environment that has been connected with AWS account and also has IAM role associated with that account. In configuration section, I defined steps how to run shell script delete-access-key.sh from directory where the file located
   

   Actions:
     delete-access-key:
       Identifier: aws/build@v1
       Inputs:
         Sources:
           - WorkflowSource
         Variables:
           - Name: IAM_USER
             Value: "tf-admin"
       Environment:
         Name: sandbox
         Connections:
           - Name: lanandra-sandbox
             Role: codecatalyst-admin
       Configuration:
         Steps:
         - Run: cd sandbox2/iam
         - Run: ./delete-access-key.sh
       Compute:
         Type: EC2
   

3. Next section will run shell script create-access-key.sh in action create-access-key. This action will depends on previous action which is delete-access-key and this action will not run if previous action is failed. More or less this action will be the same with action delete-access-key. We need to export environment variable value IAM_USER. Similar with previous action, i will input example user called tf-admin. Action will be ran in sandbox environment that has been connected with AWS account and also has IAM role associated with that account. In configuration section, there will be different steps, I defined steps to run shell script create-access-key.sh from directory where the file located. Beside that, this action will generate output artifact called ACCESSKEY, this artifact consist of file access-key.csv where information releted access key and secret key stored
   

   create-access-key:
       DependsOn:
         - delete-access-key
       Identifier: aws/build@v1
       Inputs:
         Sources:
           - WorkflowSource
         Variables:
           - Name: IAM_USER
             Value: "tf-admin"
       Outputs:
         Artifacts:
           - Name: ACCESSKEY
             Files:
               - sandbox2/iam/access-key.csv
       Environment:
         Name: sandbox
         Connections:
           - Name: lanandra-sandbox
             Role: codecatalyst-admin
       Configuration:
         Steps:
         - Run: cd sandbox2/iam
         - Run: ./create-access-key.sh
       Compute:
         Type: EC2
   

Example of Automate Access Key Rotation in CI/CD CodeCatalyst

After configure CodeCatalyst workflow file, next we verify scheduled automation access key rotation is working as expected.

1. Go to CodeCatalyst web console, choose space and project where workflow resided. Go to menu CI/CD — Workflows. Find workflow name. In section Recent runs, verify there is a successful scheduled run

   

2. Click action delete-access-key. In the right will appear window to check Logs related to that action. Verify shell script delete-access-key.sh is run and successfully found existing access key in user and delete that key

   

3. Click action create-access-key. In the right will appear window to check Logs related to that action. Verify shell script create-access-key.sh is run and successfully create access key

   

4. Go to Artifacts menu in related run. Verify artifact is successfully created and verify file can be downloaded

   

5. Open csv file inside the artifact, verify information related access and secret key. Verify user is match with what defined in workflow environment variable

   

6. For validation, go to AWS web console. Go to IAM user menu and find related user. Verify access-key is match with what defined inside csv file

   

Summary

We have reached the last section of this blog. Here are some key takeaways that can be summarized:

• There are two types of security credentials in AWS, long term and temporary
• Temporary credentials is more recommended. But in some cases or environments, we may need or have to use long term credentials such as access key
• To prevent security hole, please ensure to rotate access key periodically
• CodeCatalyst can be used as tool to schedule automation of access key rotation

Please check out my GitHub repository to see source code example that used in this blog:

• Shell scripts used for access key rotation: rotation scripts
• Amazon CodeCatalyst workflow file: codecatalyst workflow

Please comment if you have any suggestions, critiques, or thoughts.

Hope this article will benefit you. Thank you!