OpenShift secrets are used to keep the sensitive information used by the application pods. It can be used to keep the credentials, certificate files or any other sensitive information.
Here we will use very simple example of using the secret. There is a pod that runs a Java application. This application connects to the database which is running external to the application pod. It can be on another pod or outside of the OpenShift cluster. The application just needs the database username and password to connect to the database (along with other obvious details like host, port and so on). We will treat database password as the only sensitive information. The application reads the database password from the configuration file stored on the file system accessible to the application.
The simplest way to use OpenShift secret in the pod is as below.
- User creates a OpenShift secret that will contain database password.
oc create secret generic db-passwd-secret --from-literal=DB_PASS=password123
- This secret is then used in the application pod as environment variable. Below is one of the few ways to load secret as environment variables.
containers: - name: app-container image: k8s.gcr.io/busybox command: [ "/bin/sh", "-c", "env" ] envFrom: - secretRef: name: db-passwd-secret
- Container startup script reads the database password from environment variable, and writes it to the configuration file in plaintext.
- Application reads the database password from the configuration file and uses it to connect to the database.
This simple way has many security concerns. The most important being the database password is visible in plaintext as environment variable and in the configuration file. Anyone who has access to the container can see the database password.
No one can make the application secure in OpenShift if the application itself does not support handling of the sensitive data securely. In this example, keeping the database password in plaintext in the configuration file is basic security flaw in the application. Such flaws cannot be handled at the infrastructure or platform layer (for example in OpenShift). The application requires certain changes to adopt to the enhanced security model.
- There are some pre-requisite changes required in the application to remove the dependency on plaintext password. The application should be updated to allow keeping database password encrypted in the configuration file and decrypting it at runtime using the provided passphrase at the time of startup. For that, it should support a way to encrypt and decrypt the password using a passphrase. We can keep it simple by using symmetric encryption where same passphrase is used to encrypt and decrypt. There should not be any need to have plaintext password available in the application container (for example to make database connectivity checks in the readiness probe).
- This passphrase is also kept in the OpenShift secret. This is a different secret than the secret used to keep the database password.
oc create secret generic passphrase-secret --from-literal=PASSPHRASE=secretpassphrase
- Init container for loading application secrets -
- An init container is created to perform the processing of secrets.
- The OpenShift secrets
passphrase-secretare made available to init container using volume mounts.
- Init container shares the emptyDir volume with application container. This data in this volume is kept in-memory and never written to the disk (with
initContainers: volumeMounts: - name: init-secret-shared mountPath: /mnt/init-secret .... containers: volumeMounts: - name: init-secret-shared mountPath: /mnt/init-secret .... volumes: - name: init-secret-shared emptyDir: medium: Memory
- Init container’s startup script reads both the secrets from volume mounts. It copies the plaintext database password (retrieved from OCP secret
db-passwd-secret) and passphrase (retrieved from OCP secret
passphrase-secret) into a file on shared volume
- The database password and passphrase are written to the init-shared-secret volume in
$ cat /mnt/init-secret/secrets DB_PASS=password123 PASSPHRASE=secretpassphrase
- The init container completes it's execution and exits. It is no more accessible to get inside the container.
- The startup script of application container is updated to read the file from shared volume. Inside the application container, the file is available at
/mnt/init-secret/secrets. The startup script reads the database password and passphrase from the shared file.
- The application container has the
encrypt.shscript included which encrypts the plaintext database password using the passphrase.
encypt.sh --passphrase secretpassphrase passowrd123
- This command generates the output as encrypted value of the password.
- It writes it to the configuration file (in encrypted format) and deletes the secrets file
/mnt/init-secret/secrets(or makes is empty). The startup script then starts the application by passing the passphrase to it.
- The application then reads the encrypted database password from configuration file, decrypts it using the provided passphrase, and uses it to connect to the database.
- The application now supports using encrypted sensitive data implicitly.
- No password or passphrase is written to anywhere on the disk or available as environment variable.
- Init container that process the mounted OpenShift secrets is not accessible to read the files from the secret mount path once it's execution completes.
- Application container reads the password and passphrase, and deletes the file from the shared volume (in-memory) so that even if someone logs into the application container, the secret file containing password and passphrase is not accessible.
Note: Make sure the application container has permissions to delete the
/mnt/init-secret/secrets file created by init container.