AWS Controllers for Kubernetes (ACK) is a Kubernetes Custom Controller that allows you to manage AWS resources through Kubernetes manifests. By using ACK, you can easily manage AWS resources in the same lifecycle as your applications. It is suitable for scenarios where you need to create AWS resources alongside your applications for testing purposes and delete them together with the application when no longer needed. This article explains how to manage AWS resources using Kubernetes manifests. While there are various ways to manage Kubernetes manifest files, this article focuses on using helmfile for management.
Prerequisites
Please install the following on your local machine:
- eksctl
- kubectl
- helm
- helmfile
Set up the following in your AWS account. This IAM policy has the necessary permissions for the IAM Controller of ACK to create IAM resources.
- IAM policy
- https://github.com/aws-controllers-k8s/iam-controller/blob/main/config/iam/recommended-inline-policy
- name: ack-iam-controller-policy
First, let's create an EKS cluster. In this example, we will deploy it with minimal configuration using eksctl.
eksctl create cluster --name ack-helmfile
Next, create an OIDC provider, IAM Role and service account.
eksctl utils associate-iam-oidc-provider --cluster=ack-helmfile --approve
eksctl create iamserviceaccount --cluster=ack-helmfile --name=ack-dynamodb-controller --namespace=ack-system --attach-policy-arn=arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess --approve
eksctl create iamserviceaccount --cluster=ack-helmfile --name=ack-iam-controller --namespace=ack-system --attach-policy-arn=arn:aws:iam::[your AWS account ID]:policy/ack-iam-controller-policy --approve
Install ACK on the created EKS cluster. This time, we will install the IAM and DynamoDB controllers. Below is a sample configuration for the ACK controllers using helmfile.
helmfile.yaml
repositories:
- name: aws-controllers-k8s
url: public.ecr.aws/aws-controllers-k8s
oci: true
environments:
{{ .Environment.Name }}:
values:
- environments/{{ .Environment.Name }}/values.yaml
---
releases:
- name: "ack-dynamodb-controller"
namespace: "{{ .Namespace | default .Values.awsControllersK8s.namespace }}"
createNamespace: true
chart: "aws-controllers-k8s/dynamodb-chart"
version: "{{ .Values.awsControllersK8s.dynamodb.version }}"
installedTemplate: "{{ .Values.awsControllersK8s.dynamodb.installed }}"
values:
- values/dynamodb.yaml.gotmpl
- name: "ack-iam-controller"
namespace: "{{ .Namespace | default .Values.awsControllersK8s.namespace }}"
createNamespace: true
chart: "aws-controllers-k8s/iam-chart"
version: "{{ .Values.awsControllersK8s.iam.version }}"
installedTemplate: "{{ .Values.awsControllersK8s.iam.installed }}"
values:
- values/iam.yaml.gotmpl
environments/test/values.yaml
awsControllersK8s:
namespace: ack-system
region: ap-northeast-1
dynamodb:
installed: true
version: 1.1.1
iam:
installed: true
version: 1.2.1
values/dynamodb.yaml.gotmpl
aws:
region: {{ .Values.awsControllersK8s.region }}
serviceAccount:
create: false
name: ack-dynamodb-controller
values/iam.yaml.gotmpl
aws:
region: {{ .Values.awsControllersK8s.region }}
serviceAccount:
create: false
name: ack-iam-controller
Applying helmfile will create the IAM and DynamoDB controllers.
helmfile -e test apply .
kubectl get deployment -n ack-system
NAME READY UP-TO-DATE AVAILABLE AGE
ack-dynamodb-controller-dynamodb-chart 1/1 1 1 79s
ack-iam-controller-iam-chart 1/1 1 1 79s
Deploying Required AWS Resources in the Application
Now that the DynamoDB controller is ready, let's use it to create a DynamoDB resource. Here, we will deploy a simple table called "ack-dynamodb-table".
helmfile.yaml
environments:
{{ .Environment.Name }}:
values:
- environments/{{ .Environment.Name }}/values.yaml
---
releases:
- name: "ack-dynamodb-table"
namespace: default
installedTemplate: "{{ .Values.dynamodb.installed }}"
chart: "./manifests"
environments/test/values.yaml
dynamodb:
installed: true
manifests/table.yaml
apiVersion: dynamodb.services.k8s.aws/v1alpha1
kind: Table
metadata:
name: ack-dynamodb-table
spec:
tableName: ack-dynamodb-table
attributeDefinitions:
- attributeName: id
attributeType: "N"
keySchema:
- attributeName: id
keyType: "HASH"
provisionedThroughput:
writeCapacityUnits: 1
readCapacityUnits: 1
helmfile -e test apply .
Applying it will create the DynamoDB table defined in the manifest.
Deploying Role and Service Account for IRSA
To utilize DynamoDB from the application, we need to use IRSA (IAM Roles for Service Accounts). IRSA enables the association of Kubernetes service accounts with IAM roles. Specifically, it involves setting up the IAM role, appropriate trust relationships, and annotating the service account to make it usable. While eksctl can handle this through commands(as we did earlier), we will create them using ACK this time.
helmfile.yaml
environments:
{{ .Environment.Name }}:
values:
- environments/{{ .Environment.Name }}/values.yaml
---
releases:
- name: "ack-iam-role"
namespace: default
installedTemplate: "{{ .Values.iam.installed }}"
chart: "./manifests"
We will obtain the OIDC URL required for the trust relationship of the role used in service account from the EKS management console and add it to the values.yaml file.
environments/test/values.yaml
iam:
installed: true
account_id: [your AWS account ID]
oidc_url: [your EKS oidc url]
We will define the policy, role, and service account in the manifest that have permissions to access DynamoDB.
manifests/role.yaml
apiVersion: iam.services.k8s.aws/v1alpha1
kind: Policy
metadata:
name: ack-dynamodb-policy
spec:
policyDocument: |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"dynamodb:UpdateTable",
"dynamodb:UpdateItem",
"dynamodb:Scan",
"dynamodb:Query",
"dynamodb:PutItem",
"dynamodb:ListTables",
"dynamodb:GetItem",
"dynamodb:DescribeTable",
"dynamodb:DeleteItem",
"dynamodb:BatchWriteItem",
"dynamodb:BatchGetItem"
],
"Resource": [
"arn:aws:dynamodb:ap-northeast-1:{{ .Values.iam.account_id }}:table/ack-dynamodb-table"
]
}
]
}
name: ack-dynamodb-policy
---
apiVersion: iam.services.k8s.aws/v1alpha1
kind: Role
metadata:
name: ack-app-role
spec:
assumeRolePolicyDocument: |
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::{{ .Values.iam.account_id }}:oidc-provider/{{ .Values.iam.oidc_url }}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"{{ .Values.iam.oidc_url }}:aud": "sts.amazonaws.com",
"{{ .Values.iam.oidc_url }}:sub": "system:serviceaccount:default:ack-app-serviceaccount"
}
}
}
]
}
name: ack-app-role
policies:
- "arn:aws:iam::{{ .Values.iam.account_id }}:policy/ack-dynamodb-policy"
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::{{ .Values.iam.account_id }}:role/ack-app-role
eks.amazonaws.com/sts-regional-endpoints: "true"
eks.amazonaws.com/token-expiration: "86400"
argocd.argoproj.io/sync-wave: "1"
name: ack-app-serviceaccount
namespace: default
helmfile -e test apply .
Applying it will create the IAM policy, IAM role with the attached policy, and Kubernetes service account defined in the manifest.
kubectl get serviceaccount
NAME SECRETS AGE
ack-app-serviceaccount 0 7m57s
default 0 49m
Testing Access to DynamoDB
With the created service account, access to DynamoDB is now permitted from the application. Let's verify this by using the AWS CLI to access DynamoDB. First, let's try executing the PutItem operation on DynamoDB without specifying the service account.
kubectl run awscli --image=amazon/aws-cli -- dynamodb put-item --table-name ack-dynamodb-table --item '{ "id": { "N": "1" }, "value": { "S": "test" }}'
This will result in an error:
kubectl logs awscli
An error occurred (AccessDeniedException) when calling the PutItem operation: User: arn:aws:sts::xxxxxxxxxxxx:assumed-role/eksctl-ack-helmfile-nodegroup-ng-NodeInstanceRole-EVFGB3QY0C29/i-097e6311fa2cc1da1 is not authorized to perform: dynamodb:PutItem on resource: arn:aws:dynamodb:ap-northeast-1:xxxxxxxxxxxx:table/ack-dynamodb-table because no identity-based policy allows the dynamodb:PutItem action
Let's now execute PutItem on DynamoDB using the service account.
kubectl run awscli --image=amazon/aws-cli --overrides='{ "spec": { "serviceAccountName": "ack-app-serviceaccount" } }' -- dynamodb put-item --table-name ack-dynamodb-table --item '{ "id": { "N": "1" }, "value": { "S": "test" }}'
The PutItem operation is now successful.
Summary
In this article, we used AWS Controllers for Kubernetes (ACK) to manage AWS resources using Kubernetes manifests. We discussed the configuration to make the application able to utilize AWS resources and explored how to manage Kubernetes manifest files using helmfile.
Using ACK and the described workflow, it becomes easier to manage AWS resources in conjunction with the application's lifecycle, allowing for convenient creation and deletion of AWS resources alongside the application.
Top comments (0)