Fetch AWS Secrets in Your Kubernetes Realm

INTRODUCTION

It is very crucial to store our credentials securely. And this article will guide you how you can leverage the scaling of aws cloud and power of kubernetes to store and use securely your credentials in your apps deployed on kubernetes environment.
In this guide I'll be using AWS Secrets Manager to store secrets securely and external-secrets in k8s to fetch those secrets securely.

AWS Secrets Manager

AWS secrets manager is a service from AWS that you can use to store your secrets/credentials securely and then use them in your applications to access your secure resources.

Creating Secret in AWS Secret Manager

First let's create our first secret on aws.
Search Secrets manager on your aws console and open it.

You'll have this page, here you can create and store secrets.

Click on store a new secret and select your secret type. For this demo we are choosing Others type of secret. Here you can store your secrets in key value pair, or you can give it json data also by clicking on Plaintext mode.

Then click on next and give this secret a name and description(optional). Then click on next.

If you want to rotate your secrets value then turn on the Automatic rotation else leave it as it is and click on next. For this demo we are keeping optional things as default.

Now on review page review your secret and click on Store to create your secret.
Now you have your secrets stored in aws secrets manager. Its time to fetch it in your k8s pods.

Setup External Secrets in K8s

First you need to install external-secrets helm chart.
Use below commands to install external-secrets:

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets \
   external-secrets/external-secrets \
    -n external-secrets \
    --create-namespace

Now you need to create a secret in your kubernetes that will have credentials to access aws secrets manager.
Use below commands to create k8s secret:

echo -n 'KEYID' > ./access-key
echo -n 'SECRETKEY' > ./secret-access-key
kubectl create secret generic awssm-secret --from-file=./access-key --from-file=./secret-access-key -n external-secrets

You can create above secret in any namespace.

Now create a Cluster Secret Store so that our k8s secrets can fetch secrets from aws.
cluster-secret-store.yaml

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: test-secret-store
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-2
      auth:
        secretRef:
          accessKeyIDSecretRef:
            name: awssm-secret
            key: access-key
            namespace: external-secrets
          secretAccessKeySecretRef:
            name: awssm-secret
            key: secret-access-key
            namespace: external-secrets

kubectl apply -f cluster-secret-store.yaml

Now we are ready to create our first external secret (es) to fetch secrets from AWS Secret manager.
my-firs-external-secret.yaml

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-first-es # name for your es resource
  namespace: my-ns # your namespace name
spec:
  refreshInterval: 5m
  secretStoreRef:
    name: test-secret-store # clutser secret store name
    kind: ClusterSecretStore
  target:
    name: my-secret # name you want to give to your secret
    creationPolicy: Owner
  dataFrom:
  - extract:
      key: myFirstSecret #name of aws secret

kubectl apply -f my-firs-external-secret.yaml

Let's verify if our resources are deployed or not. use commands as shown in the screenshot below:

Now let's deploy a test application and fetch aws secrets.
test-app-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-app-deployment
  namespace: my-ns
  labels:
    app: test-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-app
  template:
    metadata:
      labels:
        app: test-app
    spec:
      containers:
        - name: test-app
          image: test-app:latest
          imagePullPolicy: Always
          securityContext:
            privileged: true
          envFrom:
            - secretRef:
                name: my-secret # name of the secret
          resources:
            limits:
              cpu: 500m
              memory: 500Mi
            requests:
              cpu: 100m
              memory: 100Mi

test-script.py

from os import environ
# This python script will read the values of aws secrets from os
# environment as we have deployed our app that way.

user = environ.get("myUser")
password = environ.get("myPassword")
otherSecret = environ.get("secret1")

print("my username is: ",user)
print("my password is: ", password)
print("my other secret is: ", otherSecret)

now create a docker image with test-app:latest name and test your app. It will fetch secrets value from AWS secrets manager.

Conclusion

• We have learned how to setup external secrets using helm.
• We have seen how to create and store secrets in secret manager.
• We have deployed our k8s external secret to fetch secrets value from aws.
• We have deployed a test app to see our implementaion.

Hope you liked the article and found it useful.
Feel free to ask any questions / give suggestions in the comments.

Thank You for reading the article 😊

Reference - external-secrets