Securing Kubernetes Secrets with GitOps and SOPS

GitOps is a methodology that uses Git as the source of truth for infrastructure and application configuration. This approach requires access to sensitive data, such as authentication tokens and private keys, to operate correctly. However, storing these secrets in Git represents a security vulnerability. To address this, Mozilla SOPS (Secrets OPerationS) can be used to encrypt and decrypt secrets stored in Git, ensuring they remain secure.

Overview of Mozilla SOPS

Mozilla SOPS is a command-line tool designed to encrypt and decrypt secrets stored in various formats, including YAML, JSON, ENV, INI, and BINARY. It supports integration with several key management systems (KMS) such as AWS KMS, Google Cloud KMS, Azure Key Vault, and Hashicorp’s Vault. If no KMS is available, a PGP keypair can be used instead.

Supported Key Management Systems

SOPS supports the following key management systems:

• AWS Key Management Service (AWS KMS)
• Google Cloud Key Management Service (Cloud KMS)
• Azure Key Vault
• age file encryption
• Pretty Good Privacy (PGP)

Known Limitations

Currently, Harness supports SOPS only for Helm-based applications. Additionally, SOPS does not support asymmetric keys for encrypting and decrypting secrets when using external key management systems like AWS KMS and Google Cloud KMS.

Encrypting Secrets with SOPS

To encrypt secrets using SOPS, follow these steps:

1. Install SOPS:

   brew install sops

1. Create a .sops.yaml File: This file defines the rules for encrypting secrets. For example, to encrypt files in the environment/secrets/dev directory using an age key:

   creation_rules:
     - path_regex: 'environment/secrets/dev/(.*).yaml'
       age: 'my-public-key'

1. Encrypt the Secrets:

   sops -e --in-place environment/secrets/dev/values-enc.yaml

1. Decrypt the Secrets:

   sops -d environment/secrets/dev/values-enc.yaml

Integrating SOPS with GitOps

Using SOPS with Helm

To integrate SOPS with Helm, you need to use Helm plugins that support SOPS. Here’s an example of how to configure Helm to use SOPS:

1. Install Helm and Helm-SOPS Plugin:

   helm plugin install https://github.com/jkroepke/helm-secrets.git

1. Encrypt Values File:

   sops -e --in-place values-enc.yaml

1. Use Encrypted Values in Helm:

   helm install my-app --values values-enc.yaml

Using SOPS with Argo CD

To use SOPS with Argo CD, you need to create a custom container image with SOPS and configure Argo CD to use it:

1. Create a Dockerfile:

   FROM alpine:latest

   RUN apk add --no-cache sops

   COPY . /app

   WORKDIR /app

   ENTRYPOINT ["sops", "-d"]

1. Build the Docker Image:

   docker build -t my-sops-image .

1. Configure Argo CD:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: argocd-cm
   data:
     helm.valuesFileSchemes: >-
       secrets+gpg-import, secrets+gpg-import-kubernetes,
       secrets+age-import, secrets+age-import-kubernetes,
       secrets, secrets+literal,
       https...

1. Patch the argocd-cm ConfigMap:

   kubectl patch configmap argocd-cm -n <agent namespace> --patch "$(cat argocd-cm-sops-patch.yaml)"

1. Mount the SOPS Tools:

   volumes:
     - name: helm-sops-tools
       emptyDir: {}
     volumeMounts:
       - mountPath: /helm-sops-tools
         name: helm-sops-tools

Example Configuration

Here’s an example of how to configure SOPS with age encryption in a .sops.yaml file:

creation_rules:
  - path_regex: 'environment/secrets/dev/(.*).yaml'
    age: 'my-public-key'

And here’s an example of how to encrypt a values file using age:

sops -e --in-place environment/secrets/dev/values-enc.yaml

Conclusion

Mozilla SOPS provides a robust solution for managing secrets in GitOps setups by encrypting and decrypting secrets stored in Git. By integrating SOPS with tools like Helm and Argo CD, you can ensure that your Kubernetes secrets remain secure and are only decrypted when needed. This approach aligns with the Platform Engineering philosophy of managing infrastructure and application configurations securely and efficiently.