The term of "secret" in Kubernetes is a bit deceptive. I don't want to go into details this time, because the internet is full of posts about this problem. In really short they are no real secrets, because ETCD (or any database behind Kubernetes) stores the records in plain text, without any encryption. We have a few options to solve this problem, let's start with the easiest one.
Kubernetes has official support for encrypting secret data at rest. It is disabled by default, and requires some cluster administration to enable it. The cluster admin has to define an EncryptionConfiguration and restart Kubernetes API service with the new config. It supports several en/decryption methods out of the box. It sounds easy as a flick, but there are some major issues with the solution. Firstly you have to define keys as plain text in the config (so keys should be readable for an attacker) and rotating the keys is not a trivial task. It is easy to admit that this solution is more than nothing but is far away from real secret protection.
If the built-in solution is not enough for you, you have to choose an external Key Management Service (KMS) like Hashicorp Vault and somehow inject secret resolution into your workload. There are several options, you can use a mutation webhook and create environment variables during pod creation for example, or use an extra sidecar to resolve secrets inside the container as volumes. The main advantage of this way is you have full control of where and how encrypted secrets would be converted into readable values, but all the coin has another side. The solutions are complex, and most importantly they are not transparent. Because Kubernetes secrets are just references to the real ones, you must configure everything on every single target cluster, or have to write tons of if-else in your favorite manifest generation tool.
What if I say there is something in the middle and let me introduce Trousseau. Trousseau is an open source tool that combines Kubernetes built-in envelop encryption with an external KMS.
With Trousseau you should use the same secrets as without it in your manifests. Kubernetes saves an encrypted version of secrets into ETCD but workloads can have the decrypted version transparently. Trousseau supports only Vault at the moment (others are on the roadmap).
Top comments (0)