How To Prevent Secrets From Ending Up On Developers' Machines

Even with environment variable storage offered by modern hosting platforms and secrets managers provided by every cloud, developers' machines are still littered with secrets in unencrypted text files because local development was left out of the picture.

But we can remedy this situation using dynamic secrets injection and ephemeral secrets files, and in this post, we'll be using the Doppler CLI to demonstrate how this is possible.

But before diving in, we need to solve the problem of secure and encrypted storage for secrets used in local development.

SecretOps and Local Development

Development scoped secrets simply don't exist in traditional solutions because secrets are siloed within the confines of their respective cloud or platform.

While multi-cloud capable secret managers such as HashiCorp Vault showed great promise, the prohibitively steep learning curve and unavoidable complexity involved with fetching secrets create a significant barrier to adoption as teams are left with no incentive to switch.

A SecretOps Platform builds upon the idea of centralized secrets storage but differs from existing solutions by providing a CLI and integrations for syncing secrets to any environment, machine, platform, or cloud secrets manager. It's the best of both worlds, providing a single source of truth for management while development teams choose the best secrets access method on a per-application basis.

How does this help local development? In a SecretOps world, each application has a Development environment specifically for use on developers' machines, solving the storage problem.

Using the Doppler CLI to demonstrate, let's dive in to explore the mechanics of dynamic secrets injection and ephemeral secrets files.

Dynamic Secrets Injection

The Doppler CLI uses the same secrets injection model as platforms such as Heroku and Cloudflare Workers by injecting the secrets as environment variables into the application process.

You can use a command:

doppler run -- npm start

Use a script:

doppler run -- ./launch-app.sh

Create a long-running background process in a virtual machine:

nohup doppler run -- npm start >/dev/null 2>&1 &

Use the Doppler CLI inside a Docker container:

…
FROM ubuntu

# Install Doppler CLI
RUN curl -Ls --tlsv1.2 --proto "=https" --retry 3 https://cli.doppler.com/install.sh | sh

CMD ["doppler", "run", "--", "npm", "start"]

And inject environment variables consumed by Docker Compose:

doppler run -- docker-compose up

It's also possible to pipe secrets in .env file format to the Docker CLI, where it reads the output as a file using process substitution:

docker run \
  --env-file <(doppler secrets download --no-file --format docker) \
  my-awesome-app

The same technique applies to creating a Kubernetes secret:

kubectl create secret generic my-app-secret \
  --from-env-file <(doppler secrets download --no-file --format docker)

But if a secrets file is what your application needs, we've got you covered.

Ephemeral Secrets Files

The Doppler CLI enables the mounting of ephemeral secrets files in .env, JSON, or a custom file format using a secrets template that is automatically cleaned up when the application process exits. Imagine how happy your Security team will be when they learn that secrets will never live on any developer's machines again!

To mount an .env file:

# Node.js
doppler run --mount .env -- npm start

# PHP
doppler run --mount .env -- php artisan serve

To mount a JSON file:

doppler run --mount env.json -- npm start

You can specify the format using --mount-format if the file extension doesn't map to a known format:

doppler run --mount app.config --mount-format json -- npm start

Or you can use a custom template, e.g. configure Firebase functions emulator using a .runtimeconfig.json file:

# 1. Create the template
echo '{ "doppler": {{tojson .}} }' > .runtimeconfig.json.tmpl

# 2. Mount the .runtimeconfig.json and run the emulator
doppler run \
  --mount .runtimeconfig.json \
  --mount-template .runtimeconfig.json.tmpl -- firebase emulators:start --only functions

You can even make things more secure by restricting the number of read requests using the --mount-max-reads option, e.g. caching PHP configuration which only requires the .env file to be read once:

doppler run --mount .env --mount-max-reads 1 --command="php artisan config:cache && php artisan serve"

If you're wondering what happens to the mounted file if the Doppler process is force killed, its file contents will appear to vanish instantly. The mounted file isn't a regular file at all, but a Unix named-pipe. If you've ever heard the phrase "everything is a file in Unix", you now have a better understanding of what that means.

Named pipes are designed for inter-process communication while still using the file system as the access point. In this case, it's a client-server model, where your application is effectively sending a read request to the Doppler CLI. If the Doppler CLI is force killed, the .env file (named pipe) will still exist, but because no process is attached to it, requests to read the file will simply hang. Just delete the file from your terminal, and you're good to go.

Summary

Thanks to SecretOps, we now have the workflows required to prevent secrets from ever living on a developer's machine again. All you need is dynamic secrets injection, ephemeral secrets files, and a single source of truth to pull it all together.