Getting Started with ACK RDS Controller

ACK controller for RDS is a Kubernetes Controller for provisioning RDS instances in a kubernetes native way.

ACK controller for RDS supports creating these database engines:

1. Amazon Aurora (MySQL & PostgreSQL)
2. Amazon RDS for PostgreSQL
3. Amazon RDS for MySQL
4. Amazon RDS for MariaDB
5. Amazon RDS for Oracle
6. Amazon RDS for SQL Server

Install ACK Controller for RDS

Let's start by setting up an EKS cluster. ACK Controller for RDS requires AmazonRDSFullAccess to create and manage RDS instances.
We will also create a service account ack-rds-controller for ACK Controller for RDS and attach AmazonRDSFullAccess IAM permissions to this service account with the help of IRSA.

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: ack-cluster
  region: eu-west-1
  version: "1.22"
iam:
  withOIDC: true
  serviceAccounts:
    - metadata:
        # Service account used by rds-controller https://github.com/aws-controllers-k8s/rds-controller/blob/main/helm/values.yaml#L81
        name: ack-rds-controller
        namespace: ack-system
      # https://github.com/aws-controllers-k8s/rds-controller/blob/main/config/iam/recommended-policy-arn
      attachPolicyARNs: 
      - "arn:aws:iam::aws:policy/AmazonRDSFullAccess"

managedNodeGroups:
  - name: managed-ng-1
    minSize: 1
    maxSize: 10
    desiredCapacity: 1
    instanceType: t3.large
    amiFamily: AmazonLinux2

eksctl create cluster -f ack-cluster.yaml

Once the cluster is created, install the ACK Controller for RDS using Helm chart. ACK Controller for RDS helm charts are stored in the public ECR repository.

Note: Check the latest version of the helm chart here

Authenticate to the ECR repo using your AWS credentials :

 aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
 Login Succeeded

Once you're authenticated, install ACK Controller for RDS in the ack-system namespace :

 helm upgrade \
    --install \
    --create-namespace \
    ack-rds \
    -n ack-system \
    oci://public.ecr.aws/aws-controllers-k8s/rds-chart \
    --version=v0.1.1 \
    --set=aws.region=eu-west-1 \
    --set=serviceAccount.create=false \
    --set=log.level=debug

During installation, we are setting these values in the Helm chart:

1. aws.region - AWS Region for API Calls.
2. serviceAccount.create - Setting this to false, since serviceAccount is created by eksctl during cluster creation above.
3. log.level - Setting this to debug to see detailed logs.

ACK controller for RDS installs several CRDs that allow you to provision RDS instances and related components:

1. GlobalCluster - Custom resource to create Aurora Global cluster.
2. DBCluster - Custom resource to create Amazon Aurora DB Cluster.
3. DBInstance - Custom resource to create Amazon RDS DB Instances.
4. DBClusterParameterGroup - Custom resource to create DB cluster parameter groups.
5. DBParameterGroup - Custom resource to create DB parameter groups.
6. DBProxy - Custom resource to create Amazon RDS Proxy
7. DBSubnetGroup - Custom resource to create DB Subnet Group.

Note:. Refer here for the reference of these CRDs.

Check ACK controller for RDS logs in cluster:

kubectl logs -f -l app.kubernetes.io/instance=ack-rds

Create Autora PostgreSQL Cluster with RDS Controller

To create an Aurora PostgreSQL Cluster with ACK controller for RDS, we have to first create a DB Subnet Group using DBSubnetGroup, then create a Aurora Cluster using DBCluster and add two Aurora Database instances in the cluster using DBInstance.

1. Create a DB Subnet Group test-ack-subnetgroup with all the subnets that are part of the VPC created by eksctl or use your existing subnets:
   

   apiVersion: rds.services.k8s.aws/v1alpha1
   kind: DBSubnetGroup
   metadata:
     name: "test-ack-subnetgroup"
   spec:
     name: "test-ack-subnetgroup"
     description: "Test ACK Subnet Group"
     subnetIDs:
       - "subnet-011e4c822231b65fa"
       - "subnet-001ce16b9b7c3578f"
       - "subnet-0a90579d9b066ecf7"
       - "subnet-00e6d2e90ea132c8b"
       - "subnet-0fbc22e542749c98c"
       - "subnet-080c0b1ed39b0aa91"
     tags:
       - key: "created-by"
         value: "ack-rds"
   

   Note: To get all the subnets created by eksctl, run command:
   

   eksctl get cluster ack-cluster -o json| jq -r '.[0].ResourcesVpcConfig.SubnetIds' 
   

2. Check the status of test-ack-subnetgroup DB Subnet Group :
   

   kubectl get dbsubnetgroup test-ack-subnetgroup -ojson| jq -r '.status.subnetGroupStatus'
   Complete
   

   if it returns Complete, then DB Subnet group is successfully created.

3. Create a secret to store the password of the master user :
   

   kubectl create secret generic "test-ack-master-password" --from-literal=DATABASE_PASSWORD="mypassword"
   secret/test-ack-master-password created
   

4. Create an Aurora DB Cluster test-ack-aurora-cluster using DBCluster :
   

   apiVersion: rds.services.k8s.aws/v1alpha1
   kind: DBCluster
   metadata:
     name: "test-ack-aurora-cluster"
   spec:
     engine: aurora-postgresql
     engineVersion: "14.4"
     engineMode: provisioned
     dbClusterIdentifier: "test-ack-aurora-cluster"
     databaseName: test-ack
     dbSubnetGroupName: test-ack-subnetgroup
     deletionProtection: true
     masterUsername: "test"
     masterUserPassword:
       namespace: "default"
       name: "test-ack-master-password"
       key: "DATABASE_PASSWORD"
     storageEncrypted: true
     enableCloudwatchLogsExports:
       - postgresql
     backupRetentionPeriod: 14
     copyTagsToSnapshot: true
     preferredBackupWindow: 00:00-01:59
     preferredMaintenanceWindow: sat:02:00-sat:04:00
   
   

5. Once created, check the status of DBCluster resource test-ack-aurora-cluster:
   

   kubectl describe DBCluster test-ack-aurora-cluster
   

   if the status is available then Aurora DB Cluster is successfully created.

6. After DBCluster is successfully created, add two Aurora DB Instances test-ack-aurora-instance-1 and test-ack-aurora-instance-2 to the DB cluster using DBInstance :
   

   apiVersion: rds.services.k8s.aws/v1alpha1
   kind: DBInstance
   metadata:
     name: test-ack-aurora-instance-1
   spec:
     dbInstanceClass: db.t4g.medium
     dbInstanceIdentifier: test-ack-aurora-instance-1
     dbClusterIdentifier: "test-ack"
     engine: aurora-postgresql
   ---
   apiVersion: rds.services.k8s.aws/v1alpha1
   kind: DBInstance
   metadata:
     name: test-ack-aurora-instance-2
   spec:
     dbInstanceClass: db.t4g.medium
     dbInstanceIdentifier: test-ack-aurora-instance-2
     dbClusterIdentifier: "test-ack"
     engine: aurora-postgresql
   

   When these instances are available, one of them will be a Writer Instance and other one will be a Reader Instance.

7. Check the status of DBInstance :
   

   kubectl describe DBInstance test-ack-aurora-instance-1
   kubectl describe DBInstance test-ack-aurora-instance-2
   

   

   Status available means that DB Instances are ready to use now.

8. Get the Writer Endpoint of the Aurora RDS cluster:
   

   kubectl get dbcluster test-ack-aurora-cluster -ojson| jq -r '.status.endpoint'
   
   test-ack-aurora-cluster.cluster-cncphmukmdde.eu-west-1.rds.amazonaws.com
   

9. Get the Reader Endpoint of the Aurora RDS cluster:
   

   kubectl get dbcluster test-ack-aurora-cluster -ojson| jq -r '.status.readerEndpoint'
   
   test-ack-aurora-cluster.cluster-ro-cncphmukmdde.eu-west-1.rds.amazonaws.com
   

When you have the Aurora DB Cluster provisioned, you would want to extract values such as Reader Endpoint and Writer Endpoint and configure them in the application to connect to the cluster.

ACK controller for RDS provides a way to import these values directly from the DBCluster or DBInstance resource to a desired Configmap or Secret.

Export the Database Details in a Configmap

FieldExport CRD allows you to export any spec or status field from a DBCluster or DBInstance resource into a ConfigMap.

1. Create an empty ConfigMap :
   

   kubectl create cm ack-cluster-config
   configmap/ack-cluster-config created
   

2. Create a FieldExport to import the value of .status.endpoint from test-ack-aurora-cluster DBCluster resource to the ConfigMap ack-cluster-config :
   

   apiVersion: services.k8s.aws/v1alpha1
   kind: FieldExport
   metadata:
     name: ack-cm-field-export
   spec:
     from:
       resource:
         group: rds.services.k8s.aws
         kind: DBCluster
         name: test-ack-aurora-cluster
       path: ".status.endpoint"
     to:
       kind: configmap
       name: ack-cluster-config
       key: DATABASE_HOST
   

3. Inspect the values of ConfigMap ack-cluster-config
   

   kubectl get cm ack-cluster-config -ojson|jq -r '.data'
   

   

Export the Database Details in a Secret

FieldExport CRD allows you to export any spec or status field from a DBCluster or DBInstance into a Secret.

1. Create an empty Secret :
   

   kubectl create secret generic ack-cluster-secret
   secret/ack-cluster-secret created
   

2. Create a FieldExport to import the value of .status.endpoint from test-ack-aurora-cluster DBCluster resource to the Secret ack-cluster-secret :
   

   apiVersion: services.k8s.aws/v1alpha1
   kind: FieldExport
   metadata:
     name: ack-secret-field-export
   spec:
     from:
       resource:
         group: rds.services.k8s.aws
         kind: DBCluster
         name: test-ack-aurora-cluster
       path: ".status.endpoint"
     to:
       kind: secret
       name: ack-cluster-secret
       key: DATABASE_HOST
   

3. Inspect the values of Secret ack-cluster-secret
   

   kubectl view-secret ack-cluster-secret --all
   

   

   Note: view-secret is a kubectl plugin, check out my blog on how to use this plugin.

📌 FieldExport and ACK resource should be in the same namespace.

Important - Clean up

To clean up the clusters, first disable the delete protection by setting .spec.deletionProtection to false for the test-ack-aurora-cluster DBCluster resource:

kubectl apply -f test-ack-aurora-cluster.yaml

dbcluster.rds.services.k8s.aws/test-ack-aurora-cluster configured

Now, you can proceed to delete the instances first and then cluster:

kubectl delete -f test-ack-aurora-instance.yaml
kubectl delete -f test-ack-aurora-cluster.yaml