Today, preserving and securely storing secrets like API keys, passwords, and certificates is an essential part of creating and deploying applications.
Google Secret Manager provides a powerful and centralized solution for secret management, as well as a secure vault to store and access sensitive information.
However, getting these secrets and seamlessly integrating them into applications might be difficult. This post will detail this process and demonstrate how simple it is to consume secrets from Google Secret Manager into a GKE cluster.
Before we get started there are couple of pre-requisites:
- Make sure the GKE cluster which you are creating should have WORKLOAD_IDENTITY enabled.
- Make sure you have a IAM SERVICE_ACCOUNT created with proper permissions.
- One which we are most interested in is "Secret Manager Secret Accessor(roles/secretmanager.secretAccessor). Refer this for more details.
- gcloud command line gives a great way to quickly create and destroy resources on GCP.
- In below command, replace
CLUSTER_NAMEwith the name of the GKE cluster,
PROJECT_IDwith the name of the project where you want to create the GKE cluster,
SERVICE_ACCOUNTwith the name of the service account which contains the necessary permissions to interact with GCP resources (for example in this case it would be Google Secret Manager),
MACHINE_TYPEwith the size of machine to deploy your workloads since we would be installing couple of resources make sure you choose a medium machine,
ZONEwith the name of the zone,
NUMBER_OF_NODESwith an integer value stating minimum number of nodes per zone,
CLUSTER_VERSIONwith the version of the GKE cluster you want to install.
gcloud container clusters create CLUSTER_NAME --workload-pool=PROJECT_ID.svc.id.goog --service-account=SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com --zone ZONE --machine-type MACHINE_TYPE --num-nodes=NUMBER_OF_NODES --cluster-version=CLUSTER_VERSION
gcloud container clusters create test-k8s \ --workload-pool=proj-101.svc.id.goog \ --email@example.com \ --zone us-central1-a \ --machine-type e2-standard-4 \ --num-nodes=1 \ --cluster-version=1.23.17-gke.5600
- Install the Secret Store CSI Drivers, the simplest way is to get them installed via
- Additionally, if you want to create secret k8s resource as well make sure to enable syncSecret flag.
helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts helm upgrade --install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver --namespace kube-system
- Install the GCP provider plugin.
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp/main/deploy/provider-gcp-plugin.yaml
- Make sure your service-account is binded with IAM GCP account
- Use below command to add/check IAM policy binding.
- Make sure to follow proper naming convention for service-account i.e. [NAMESPACE/SERVICE_ACCOUNT_NAME] for instance here we have [default/app-sa].
gcloud iam service-accounts add-iam-policy-binding \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:$proj-101.svc.id.goog[default/app-sa]" \ firstname.lastname@example.org
Once all these things are done you are ready to use the secrets from Google Secret Manager and consume them in your deployments.
- Use the same GCP IAM service account in k8s service account annotations.
--- apiVersion: v1 kind: ServiceAccount metadata: annotations: iam.gke.io/gcp-service-account: email@example.com name: app-sa
- Create the k8s
SecretProviderClassresource and have your path of secret appended to resourceName key (this is the one which will come from Google Secret Manager).
- In this example, I already created a secret called testpassword in GSM.
--- apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: test-secret spec: provider: gcp parameters: secrets: | - resourceName: "projects/proj-101/secrets/testpassword/versions/latest" fileName: "testpassword" secretObjects: - secretName: testpassword data: - key: testpassword objectName: testpassword type: Opaque
- Create the pod/deployment and use your secrets in your application.
- Here we are creating a volume item which is referring secrets-store.csi.k8s.io csi drivers and using the secretProviderClass which we created in above step.
- Further, we are mounting this as a volume to be consumed.
- Note: If you are planning to use the secret as an environment variable make sure the secret (k8s resource) is present.
--- apiVersion: v1 kind: Pod metadata: name: test-pod spec: serviceAccountName: app-sa containers: - image: gcr.io/google.com/cloudsdktool/cloud-sdk:slim name: mypod resources: requests: cpu: 100m tty: true volumeMounts: - mountPath: "/var/secrets" name: vol-secret readOnly: true volumes: - name: vol-secret csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "test-secret"
I hope you find this post useful, feel free to add your thoughts or comments.