In this article, I invite you to explore a solution I developed that leverages the Keeper Secrets Manager SDK to provide a REST API for retrieving credentials from the Keeper Enterprise Password Manager platform.
In a Continuous Integration/Continuous Deployment (CI/CD) environment, the automation of building, testing, and deployment of applications requires access to various secrets, such as API keys, database passwords, and credentials for third-party services. And although CI/CD platforms offer some sort of secrets management functionality, maintaining these secrets is still a pain point for teams.
Leveraging the Keeper Secrets Manager SDK
One of the problems that I faced using Keeper Secrets Manager, the CI/CD app for Keeper Enterprise Password Manager, is that I needed to install it on each run of the pipelines. Further complicating things, when using Docker agents, which resets the build environment even between stages, the installation process can be cumbersome because images were to minimalistic and lacked dependencies which I had to track down before being able to install the app itself. I ended up creating custom base images for the Gitlab runners, but when colleagues needed to use their own images they were forced to revert to the install process and we ended up in the same dependency hell.
Luckily, the Keeper team provides several SDKs, one of which is for Go. As such, I came up with the idea of building a Go app that would expose a REST endpoint, and using the Keeper Secrets Manager SDK I could connect to the Keeper Enterprise Password Manager to retrieve the secrets.
The idea opened up several usage scenarios, one of which is adding the app as a service container to the Gitlab runners that will be available to be queried without the need to install anything else. Just provide it the KSM_CONFIG data, either as an environment variable in the container or pass it through a request header, and it will connect to the correct Keeper Enterprise Password Manager vault.
Prerequisites
Unfortunately, at the time of writing, Keeper Secrets Manager is available only for Business accounts.
To configure a Keeper Secrets Manager application to access your Keeper Enterprise Password Manager secrets, you can follow the excellent tutorial on the Keeper's website: https://docs.keeper.io/secrets-manager/secrets-manager/quick-start-guide
KSM-Api wrapper
To begin using my project, the first step is cloning the Github repository onto your local machine:
git clone https://github.com/Hexagonal-Software/ksm-api.git
Since the project is developed in Go, you'll need to download its dependencies after cloning:
cd ksm-api
go mod download
You have the option to run the application by building it as an executable or packaging it in a Docker image. I am using it as a standalone daemon on servers where I ran build pipelines directly on the machine, and use a Docker image where the build environment uses Docker agents. In a later section I will provide a configuration for running it as a service container for Gitlab runners.
The project includes multiple Make commands to streamline the build process.
For creating a Docker image with the KSM-Api executable use:
make release-docker OPTS="--platform=linux/amd64" REPO="denisrendler/kms-api"
When the release-docker command completes you can find the image in your local Docker instance. Note that the image will be tagged with the latest code tag retrieve from the Git repository, which is 0.1.0 at the time of this writing. This is added automatically by the make command.
To run the app in a Docker container, you must configure the KSM_CONFIG data as an environment variable when creating the container:
export KSM_CONF="W2FkZCB5b3UgYmFzZTY0IGVuY29kZWQgS2VlcGVyIGFwcGxpY2F0aW9uIGNvbmZpZyBkYXRhIGhlcmVdCg=="
docker run --name=ksm-api -ti -p 8086:8086 -e "KSM_CONFIG=${KSM_CONF}" denisrendler/ksm-api:0.1.0
With the provided KSM_CONFIG data, you can access any secrets managed by the Keeper Secrets Manager using a simple curl request:
curl --globoff 'ksm-api:8086/api/v1/secret/<record UID or name>/<field OR custom_filed>/<field name>'
Additionally, you can pass the KSM_CONFIG data as a request header:
export KSM_DATA="W2FkZCB5b3UgYmFzZTY0IGVuY29kZWQgS2VlcGVyIGFwcGxpY2F0aW9uIGNvbmZpZyBkYXRhIGhlcmVdCg=="
curl -H "KSM_CONFIG: ${KSM_DATA}" --globoff 'ksm-api:8086/api/v1/secret/<record UID or name>/<field OR custom_filed>/<field name>'
The path parameters mirror those used by the Keeper Secrets Manager app's secret notation subcommand, as described in the following table:
| Parameter | Description | Example |
|---|---|---|
| record UID or name | The UID or the name of the secret. I recommend using the name because it makes it easier when switching between environments. | oxhtLx9qrQIzeSXBtvQj2Q my_keeper_secret |
| field or custom_field | Use field if the data you want to retrieve is part of the secret's type, ie. password when it is a login or host[hostName] when it is a server type.Use custom_field when the data is stored inside a field type that is not part of the secret's template, ie. the field was added using the Add Custom Field button. |
field |
| field name | The field name you want to retrieve. | password host[hostName] host[port] |
Configure KSM-Api as a Gitlab service container
The easiest way to have the service container always available is to edit your runner configuration and add the following lines:
[[runners.docker.services]]
name = "denisrendler/kms-api:0.1.0"
alias = "ksm-api"
Now, define a KSM_CONFIG environment variable and add your base64 config data. Make sure to uncheck the Protect variable checkbox to make the variable available to all pipelines.
By default, Gitlab will not automatically pass user defined environment variables to service containers. To pass your KSM_CONFIG data from the CI/CD variable, you must explicitly define it in your .gitlab-ci.yml:
variables:
KSM_CONFIG: ${KSM_CONFIG}
That is it! Whenever you need to retrieve a secret you can use a curl request to get it.
Design decisions
There are a couple of design decisions that you need to be aware of:
- the application does not handle TLS connections since it is build to be run in a limited environment on a local system or containerised environment. This way the app avoids issues with requests not handling self-signed certificates.
- error messages are only available in the logs as a precaution to avoid dumping sensitive information in CI/CD logs.
- since this is an external application, CI/CD platforms might not obfuscate or otherwise hide sensitive information if dumped into the logs. Also, you will need to take care of workspace cleanup after running the pipeline if you dump to disk information like SSH keys.
If you find a use case where these decisions might have a bigger impact, open an issue on the Github repository and we can take it into account to add new functionality.
Contributing
The project started from an idea I had to resolve a pain point I encountered. But, if you feel that further functionality can bring more value to the app I am open to suggestions and even code contributions.
Just open a new issue or pull request on the Github repository and let's discuss it.
Bug reports are also welcomed.
Wrapping up...
While Keeper Enterprise Password Manager is a great password manager with an excellent zero-knowledge architecture, for me one of the more tedious aspects of managing CI/CD pipelines has been to maintain the Keeper Secrets Manager (KSM) app installation, a process that not only introduced potential points of failure but it has also deterred adoption for many of my colleagues.
My project addresses this pain point by integrating seamlessly with Keeper Enterprise Password Manager through the Keeper Secrets Manager SDK, and using it as a service container it eliminates the need for repetitive installations. It offers a streamlined, secure way to manage and access secrets across the pipelines, significantly reducing build times and enhancing the security and reliability of CI/CD processes. This solution not only simplifies the developer's workflow but also ensures that your secrets are always managed efficiently and securely.
Top comments (0)