Vault Associate Certification (Part 2): Create Vault policies

In the previous post in this series we looked at authentication methods. Once we have authenticated, what are we allowed to do in Vault? This is where policies come into the picture.

Policies are attached to Vault tokens and specify what the given Vault token can do in Vault. A policy consists of a collection of capabilities on a number of paths in Vault. An example of a capability is read and an example of a path is secret/data/database/password. If I receive a token with a policy that has the read capability on the secret/data/database/password path I will be able to read the information stored on that path. Intuitive, isn't it?

In the previous post I enabled auth methods without explicitly authenticating to Vault first. In policy terms I was performing the create capability on various auth/... paths. How was I allowed to do that? I was implicitly using the root token¹, and this token has the root policy attached. The root policy allows you to do anything you want in Vault. Generally we do not want all users to be able to do whatever they want in Vault, so that is the motivation to why we need policies.

Policies make up the second objective in the Vault certification journey. This objective covers the following sub-objectives:

• Illustrate the value of Vault policy
• Describe Vault policy syntax: path
• Describe Vault policy syntax: capabilities
• Craft a Vault policy based on requirements

Note that there are four sub-objectives listed, but I have combined two of them since they are so closely related that it is difficult to talk about one without talking about the other.

Let's get started!

Illustrate the value of Vault policy

If you authenticate to Vault using any of the enabled authentication methods you receive a token upon successful authentication. Tokens allow you to further interact with Vault. However, there is a big if attached to that last statement. Your token must have an attached policy, or several policies. These policies in turn determines if the API-calls you send to Vault will respond with a 2XX response code indicating success, or a 403 response code indicating that you are not allowed to perform the action.

Policies are deny by default. If you have no policy that allows you do to action-X, then you will not be able to do action-X. This is a good thing, it makes it easier to write policies to explicitly allow certain actions instead of having to think of all the things you don't want to allow a user to do.

It is possible to have multiple policies attached to your token. Policies can be attached directly from the authentication method that provided you with your token, or they could come from group memberships, or even from a concept known as entities in Vault. More on entities in a future post, but for now just think of an entity as your identity inside of Vault.

Policies allow Vault administrators to delegate permissions to other users and apps to perform actions in Vault:

• An application can be given permissions to read and write to a given secret path.
• A user can be given permissions to handle the administration of the GitHub auth method.
• A CI/CD system can be given permissions to read deployment credentials from Vault.

This is the value that policies bring. Restricting who can do what in Vault. It is not much more to say about it!

Describe Vault policy syntax: path and capabilities

Policy documents are written in HashiCorp Configuration Language (HCL) or in JSON. I prefer HCL, but keep in mind that JSON is available just in case a question about this comes up in the exam.

A basic policy document named policy.hcl looks like this:

// policy.hcl
path "secret/data/database/password" {
    capabilities = [ "read" ]
}

This policy document contains a single path block, but in general a policy document will contain multiple path blocks. The single path block in this document has a single capability listed in capabilities: the read capability. Note that capabilities is always a list even if there is only a single entry.

Everything in Vault exists at a path. As we saw in the previous post auth methods exist at auth/<auth method> paths. Configuration related to the Vault server itself exists at the sys/ path. Secrets are stored at various paths depending on what secrets engine we use. What we are allowed to do at a given path is determined by the capabilities for that path in the policy or policies that our token has.

What capabilities exist? The supported capabilities are:

• create allows creating data at a path. This capability corresponds to POST and PUT HTTP verbs in the Vault REST API.
• read allows reading data stored at a path. Corresponds to GET requests in the API.
• update allows changing the data stored at a path. This capability also corresponds to POST and PUT requests in the API.
• patch allows partial updates of the data stored at a path, without completely changing all the data. Corresponds to PATCH requests in the API.
• delete allows deleting the data stored at a path. Corresponds to DELETE requests in the API.
• list allows listing values at a path, i.e. sub-paths that exist under the path. Does not allow listing the data stored at the path, e.g. secret values.
• deny disallows access. This capability is mostly used in combination with a more open policy statement that might give you access to a broad set of paths, but you then want to remove one or two paths from that. Then the deny capability for these paths allows you to do that.
• sudo allows access to root-protected paths. This capability is used in addition to other capabilities needed for the specific path. Usually paths under sys/ are root protected.

As is clear from the list above, most capabilities correspond to some HTTP verb in the API. However, the capabilities make it more clear what actions you are permitting than thinking of them as HTTP verbs.

Using these new capabilities we learnt about we can construct a new policy:

// policy2.hcl
path "secret/data/database/password" {
    capabilities = [ "create", "read", "update" ]
}

path "secret/data/database/username" {
    capabilities = [ "read" ]
}

This policy allows us to read the username stored at secret/data/database/username and create/read/update the password at secret/data/database/password.

There are two concepts that allow us to write a bit more dynamic policies. These are the wildcard character + and the glob character *. You might be familiar with the glob character from other languages, the purpose of it is to match any number of characters. It can only be used at the end of a path. Some examples:

• The path secret/data/database/* matches any path that starts with secret/data/database/, for instance secret/data/database/password or secret/data/database/postgres/westeurope/password.
• The path secret/api-* matches any path starting with secret/api-, for instance secret/api-key or secret/api-keys/foo.

The glob character can only be used at the end of a path, so a path such as secret/*/dev is not valid.

The wildcard character is used to denote any single complete path segment. A path segment is the part between two / characters. Some examples:

• The path secret/data/+/dev matches secret/data/api/dev, secret/data/database/dev, etc.
• The path secret/data/+/dev/+/key matches secret/data/api/dev/azure/key, etc.
• The path secret/data/+/database matches secret/data/dev/database, secret/data/prod/database, etc.

Apart from the wildcard and glob characters there are additional ways of creating dynamic policies. To understand these I need to introduce the concept of entities and the identity secrets engine. I will postpone that discussion until part five of this series.

We can now extend our example policy a bit further:

// policy3.hcl
path "secret/data/database/*" {
    capabilities = [ "list" ]
}

path "secret/data/+/api" {
    capabilities = [ "create", "read", "update", "patch" ]
}

path "secret/data/database/password" {
    capabilities = [ "create", "read", "update" ]
}

path "secret/data/database/username" {
    capabilities = [ "read" ]
}

In addition to what we had before this new version of the policy also allows us to list the paths available below the secret/data/database/ path. I also added permissions for creating and using secrets at the secret/data/+/api path, e.g. secret/data/dev/api and secret/data/prod/api.

The root token is special. It has the root policy attached to it. I don't think you can actually list the contents of the root policy, but you can think of it conceptually as the following policy document:

// root-policy.hcl
path "*" {
    capabilities = [ "read", "create", "list", "patch", "update", "sudo", "delete" ]
}

It is clear that we want to be careful with the root token due to this permissive policy!

Craft a Vault policy based on requirements

This sub-objective is a bit vague, but really what it means is that given a user or an application that we know needs to do a given task in Vault - write a policy for that.

When crafting a policy it is immensely helpful to use the -output-policy flag in the Vault CLI. Use this flag with the command you want the policy to allow and you will get a sample policy as output that shows how to write the policy to allow the action. I will use this flag in the examples below.

The generic Vault example is that we have an application that requires access to secrets. Let's imagine that we also want the application to be able to rotate the secret value. The policy must allow:

• Read a secret database connection-string from secret/data/database/connection-string.
• Update of the secret value stored at secret/data/database/connection-string.

This is a simple policy so we can write it without any help, but for the sake of it let us use the -output-policy flag. For this example to work I have to first start my Vault dev server:

$ vault server -dev

Next I make sure that I have a key/value secrets engine² enabled:

$ vault secrets list

Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_5a885e2b    per-token private secret storage
identity/     identity     identity_7c0fc8ae     identity store
secret/       kv           kv_504faf32           key/value secret storage
sys/          system       system_d5310253       system endpoints used for control, policy and debugging

I can see the kv secrets engine mounted at the path secret/, this is what I need!

To read a secret at secret/data/database/connection-string I run the following command, note that at this point the secret at this path does not even exist:

$ vault kv get -output-policy -mount=secret database/connection-string

path "secret/data/database/connection-string" {
  capabilities = ["read"]
}

There we have it, a short policy with the details we need! The next thing our policy had to allow was updating the value stored at this same path:

$ vault kv put -output-policy -mount=secret database/connection-string connection_string="secret"

path "secret/data/database/connection-string" {
  capabilities = ["create", "update"]
}

The output included both the create and the update capability. We could include the create capability if we also want the application to be able to create the secret to start with, but the requirements did not say that this was necessary so we should skip it this time. If we combine our two outputs we end up with the following policy:

// policy.hcl
path "secret/data/database/connection-string" {
  capabilities = ["read", "update"]
}

This was a simple scenario but it illustrates the workflow of constructing a policy from given requirements.

That was the end of my post on Vault policies. In future posts we will see how policies are attached to tokens, and this is when the restrictions imposed by the policies actually happen. Until then write a few policies for some use-case to get hands-on practice!

1. More on root tokens, and tokens in general, in part three of this series. ↩

2. More on secrets engines and how you work with them in part five of this series. ↩