Vault Associate Certification (Part 3): Assess Vault tokens

So far in this course we have covered auth methods and policies. We use auth methods to obtain Vault tokens, and attached to these tokens are policies that put restrictions on what they can do. Tokens make up the core authentication method inside of Vault. In this post it is time to take a closer look at the topic of tokens.

Tokens make up the third objective in the Vault certification journey. This objective covers the following sub-objectives:

• Describe Vault token
• Differentiate between service and batch tokens. Choose one based on use-case
• Describe root token uses and lifecycle
• Define token accessors
• Explain time-to-live
• Explain orphaned tokens
• Create tokens based on need

There are many sub-objectives to look at in this post, some of them small in scope and some of them a bit larger. One thing to note about this post is that it will focus on the token auth method and how we work with tokens that come from this auth method. Similar settings as we configure for these tokens can be configured for tokens that come as the result from other auth methods.

Let's dig in!

Describe Vault token

A Vault token is used together with the token auth method to authenticate actions in Vault. We have previously seen that we use other auth methods in order to obtain Vault tokens, and then we use the tokens for further interactions with Vault.

To get started talking about Vault tokens, let's start a Vault development server¹ and run a command to describe our current token:

$ vault token lookup

Key                 Value
---                 -----
accessor            dhygs9oMFq7iz720DTYk3T3Q
creation_time       1693559414
creation_ttl        0s
display_name        root
entity_id           n/a
expire_time         <nil>
explicit_max_ttl    0s
id                  hvs.aytOgHUVgI8o4CgI0cycLHWk
meta                <nil>
num_uses            0
orphan              true
path                auth/token/root
policies            [root]
ttl                 0s
type                service

A token has a number of properties. Later on in this post I will talk about the accessor, ttl/creation_ttl/explicit_max_ttl, orphan and type properties. For this token we see that it was created at a given time as specified in the creation_time property. It has a name of root as specified in the display_name property. We also see that it has one policy attached, and that is the root policy.

The actual token value itself is seen in the id property. Here my token is hvs.aytOgHUVgI8o4CgI0cycLHWk. If I run the following command we see that I get the same output as before:

$ vault token lookup hvs.aytOgHUVgI8o4CgI0cycLHWk

Key                 Value
---                 -----
accessor            dhygs9oMFq7iz720DTYk3T3Q
creation_time       1693559414
creation_ttl        0s
display_name        root
entity_id           n/a
expire_time         <nil>
explicit_max_ttl    0s
id                  hvs.aytOgHUVgI8o4CgI0cycLHWk
meta                <nil>
num_uses            0
orphan              true
path                auth/token/root
policies            [root]
ttl                 0s
type                service

The vault token lookup command defaults to the current token that is configured in the VAULT_TOKEN environment variable or in the ~/.vault_token file.

A few of the things you can configure for a token include:

• How many times the token can be used before it expires.
• For how long time the token is valid.
• What policies the token should have.

We will see examples of this in the following sections.

Differentiate between service and batch tokens. Choose one based on use-case

So far I have used the generic term token for any kind of token. There are in fact two distinct types of tokens: service tokens and batch tokens.

Service tokens are the default tokens, these tokens support all the features of tokens that will be discussed throughout this post and the other posts in this series. Service tokens require Vault to read and write to its storage backend, and when the number of existing service tokens grow this can become a problem for Vault and might require you to scale up your machines running Vault. This is typical for systems where you have many Vault clients making a lot of read and write requests continuously.

Batch tokens are not stored in memory so it does not require any interaction with the storage backend. The tokens themselves carry enough information to be used to perform actions in Vault. However, they lack many of the features that service tokens have. Batch tokens appear as BLOBs, or Binary Large Objects.

Below follows a short comparison between service and batch tokens:

It is clear from the comparison above that batch tokens lack many of the features that you might want to use. There are additional differences between service tokens and batch tokens, but the list above covers the most prominent differences.

Given a token value you can immediately say which type of token it is. The rules are as follows:

• A service token value starts with hvs.
• A batch token value starts with hvb.

To illustrate this, let's first create a service token:

$ vault token create -type=service -policy=default

Key                  Value
---                  -----
token                hvs.CAESILfkHZ292kPHfJ<truncated>lESXBMdWxsdnU
token_accessor       tyygHCNyV2RBxSl91TkDLvOz
token_duration       768h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

The token value is hvs.CAESILfkHZ292kPHfJ<truncated>lESXBMdWxsdnU, so we know that it is a service token since it starts with hvs..

Now let's create a batch token:

$ vault token create -type=batch -policy=default

Key                  Value
---                  -----
token                hvb.AAAAAQK3fkmSCvwf<truncated>ZfIPtD10ayUNBfml
token_accessor       n/a
token_duration       768h
token_renewable      false
token_policies       ["default"]
identity_policies    []
policies             ["default"]

The output tells us that our token hvb.AAAAAQK3fkmSCvwf<truncated>ZfIPtD10ayUNBfml is a batch token since it starts with hvb..

Note that the -type flag in the commands above defaults to service, so we do not need to include it if we want to create service tokens.

Describe root token uses and lifecycle

When you create your Vault server for the first time you receive a root token. A root token is any token that has the root policy attached to it. The root policy allows the token to perform any action in Vault.

HashiCorp's production hardening guide for Vault specifies the following recommendation concerning root tokens:

Avoid Root Tokens. Vault provides a root token when it is first initialized. This token should be used to setup the system initially, particularly setting up auth methods so that users may authenticate. We recommend treating Vault configuration as code, and using version control to manage policies. Once setup, the root token should be revoked to eliminate the risk of exposure. Root tokens can be generated when needed, and should be revoked as soon as possible.

Commit this recommendation to memory. On the exam there could be questions related to the handling of root tokens, and the answer should align to this recommendation. To summarize this sub-objective in two short points:

• Use a root token to initialize Vault with auth methods and policies
• Revoke the root token after initialization, generate a new token if required

If you must keep a root token alive for longer periods you should keep it safe in a separate system, perhaps in a password manager protected by a strong password and two-factor authentication.

Define token accessors

We saw earlier that one of the properties on a token was the token accessor. Think of the token accessor as a pointer to the token that allows you to perform a few operations on the token, without knowing the value of the actual token. These actions include:

• Running vault token lookup -accessor=<accessor> to see properties of the token, except for the token ID.
• Renewing a token using vault token renew -accessor=<accessor>.
• Revoking a token using vault token revoke -accessor=<accessor>.

The purpose of the token accessor is thus to provide a way to delegate the administration of the token, without giving away the token value itself.

Explain time-to-live

A few of the metadata properties we saw for tokens at the beginning of this post concerned TTL, or time-to-live. The token TTL determines for how long, expressed as the number of hours, minutes, and seconds, the token will be valid for. When the TTL passes, the token is no longer valid.

One exception to this is the initial root token that does not have a time-to-live, it is valid as long as it is not disabled.

The default TTL for a token is 32 days, or 768 hours. This is a fairly long TTL, so you might want to restrict it a bit. We can generate a token with a shorted TTL by running the following command:

$ vault token create -ttl=10h -policy=default

Key                  Value
---                  -----
token                hvs.CAESID<truncated>VNbnhsNVE
token_accessor       1GwXHiYUblIJETPc821GnbpB
token_duration       10h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

In this example I created a token with a TTL of ten hours (10h). The TTL is expressed with a string such as:

• 60s (or just 60) for sixty seconds
• 10d for ten days
• 20m for twenty minutes

For the full details of how to specify a time duration see the official documentation.

The token I created above got the TTL of ten hours, but let's lookup the token and see all of its properties:

$ vault token lookup hvs.CAESID<truncated>VNbnhsNVE

Key                 Value
---                 -----
accessor            1GwXHiYUblIJETPc821GnbpB
creation_time       1693853984
creation_ttl        10h
display_name        token
entity_id           n/a
expire_time         2023-09-05T06:59:44.816032+02:00
explicit_max_ttl    0s
id                  hvs.CAESID<truncated>VNbnhsNVE
issue_time          2023-09-04T20:59:44.816034+02:00
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
policies            [default]
renewable           true
ttl                 9h59m53s
type                service

The creation_ttl is 10h, and the current ttl is 9h59m53s. However, we see that explicit_max_ttl is 0s, which means that it is infinitely renewable because there is no explicit max TTL. This might not be the behavior we want, so let us rectify the situation:

$ vault token create -ttl=10h -explicit-max-ttl=100h -policy=default

Key                  Value
---                  -----
token                hvs.CAESIOkA<truncated>V0aURuNURLZEc
token_accessor       tHQlGisrwbdkar5cPN2ekFVe
token_duration       10h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Then let's run a lookup on this token:

$ vault token lookup hvs.CAESIOkA<truncated>V0aURuNURLZEc

Key                 Value
---                 -----
accessor            tHQlGisrwbdkar5cPN2ekFVe
creation_time       1693854190
creation_ttl        10h
display_name        token
entity_id           n/a
expire_time         2023-09-05T07:03:10.315457+02:00
explicit_max_ttl    100h
id                  hvs.CAESIOkA<truncated>V0aURuNURLZEc
issue_time          2023-09-04T21:03:10.315458+02:00
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
policies            [default]
renewable           true
ttl                 9h59m10s
type                service

This looks better, now the explicit_max_ttl is set to 100h which means that at most this token will live for one hundred hours. It will not be possible to renew the token further than this.

In the table above where we looked at differences between service tokens and batch tokens we saw that a service token can be periodic. What does this mean? A periodic token is a token that can be renewed within a given window of time.

Creating a periodic token is an operation that requires the sudo capability on the auth/token/create path. Remember capabilities and paths for policies from the last post? If not, go back and read that part again now!

The property that distinguishes a periodic token from other tokens is that it has no max TTL. In fact, this is similar to the token we created above where we had a TTL but set the explicit max TTL to 0s. Although similar they are not really the same, but for now let us ignore this technical detail.

As long as the periodic token is renewed before its TTL expires it keeps on being valid for another window of time. If it is not renewed, it will expire.

Creating a periodic token using the CLI means adding the -period flag to the token creation command:

$ vault token create -period=24h -policy=default

Key                  Value
---                  -----
token                hvs.CAESIHJWD<truncated>jRHeEdiTmo
token_accessor       09LI02728ZQApPF48Npaqh0a
token_duration       24h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

The response indicates that the token is renewable (token_renewable is true), and that the duration the token is valid is one day (token_duration is 24h). If we run a token lookup command on the token we see the following:

$ vault token lookup hvs.CAESIHJWD<truncated>jRHeEdiTmo

Key                 Value
---                 -----
accessor            09LI02728ZQApPF48Npaqh0a
creation_time       1693763013
creation_ttl        24h
display_name        token
entity_id           n/a
expire_time         2023-09-04T19:43:33.390499+02:00
explicit_max_ttl    0s
id                  hvs.CAESIHJWD<truncated>jRHeEdiTmo
issue_time          2023-09-03T19:43:33.390501+02:00
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
period              24h
policies            [default]
renewable           true
ttl                 23h59m16s
type                service

The explicit_max_ttl property is 0s which means there is no explicit max TTL. We can also see that period is 24h and the current ttl is 23h59m16s.

Explain orphaned tokens

Tokens created using the token auth method make up a hierarchy. This means that a token that is used to create another token becomes the parent of this token, and the created token becomes a child token. An example of a token hierarchy:

(root)
├── token-a
│   └── token-b
├── token-c
└── token-d
    └── token-e
        └── token-f

token-a is the parent of token-b, and token-d is the parent of token-e. token-e is in turn parent of token-f. token-c has no child tokens. All tokens are descendants of some root token (not necessarily a root token, here I mean a root of all the tokens in this hierarchy).

A child token expires if its parent token expires. This means a while branch in a hierarchy might expire at the same time if the token at the root of that branch expires. This is a good feature as long as you are aware of it and use it correctly. If you are unaware of it this will come as a nasty surprise.

An orphaned token is a token that has no parent token. If you have no parent token, then you don't expire when your parent token expires. The orphaned token only expires whenever its own time-to-live expires, or when it is revoked explicitly. We can create an orphan token using the following command:

$ vault token create -policy=default -orphan

Key                  Value
---                  -----
token                hvs.CAESIGG8z0EzIMKu1H0XOfbPQ-uckV7D6oUnuDK6syPBxhTOGh4KHGh2cy56NUNTMW9FaUlKeEMxRjdVWW9DbEZrSG4
token_accessor       8H0VPFuRufLg4AYjApzN5bF4
token_duration       768h
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]

To be allowed to create an orphaned token you must fulfill one of the following:

• If you have write access to the auth/token/create path. This path also requires the sudo capability. The required policy is:
  

  path "auth/token/create" {
      capabilities = [ "create", "update", "sudo" ]
  }
  

• If you have write capability on the auth/token/create-orphan path. If you use the CLI you must run the vault write -f auth/token/create-orphan @payload.json command to hit this endpoint, if you use vault token create -orphan it will instead use the auth/token/create path which, as noted above, requires sudo privileges. The required policy is:
  

  path "auth/token/create-orphan" {
      capabilities = [ "create", "update" ]
  }
  

• Technically you also generate orphaned tokens whenever you authenticate with an auth method that is not the token auth method.

Create tokens based on need

This section will repeat some of the concepts from previous sections, but it will focus on using the Vault CLI to create various tokens. What type of token you require depends on your needs. Sometimes it is not clear what your needs are.

Create a periodic token

A periodic token is a token that does not have an explicit max TTL. It can be renewed within a given period, and if so it is valid for another period of time. If this period is set to one hour it means that the initial token is valid for one hour, and if it is renewed within that hour it is valid for one more hour starting when the token was renewed.

A periodic token is created with the following command:

$ vault token create -period=1h -policy=my-policy

A periodic token is useful if you have an application or some system that is not able to interact with Vault and handle generating new tokens. This could be a legacy system where you have limited capabilities to update the code, or maybe some third-party system with limited functionality that you have no influence over. In these situations you might still be able to make a request to Vault to have the token renewed, so that you then can keep on using the same token.

Create an orphaned token

An orphaned token is a token that does not have a parent token. A consequence of that is that it does not expire when its parent expire (since it does not have one!)

An orphaned token is created with the following command:

$ vault token create -orphan -policy=my-policy

An orphaned token is useful if you want to make sure your token only expires when the TTL is up, and not when a parent token that you have no control over somewhere in the token hierarchy expires.

Create a batch token

A batch token carries enough information to be used by itself and it is not stored in memory in Vault. It is used in scenarios where you need a low impact on your Vault server, which could be the case if you have a large number of unexpired service tokens.

A batch token is created with the following command:

$ vault token create -type=batch -policy=default

Key                  Value
---                  -----
token                hvb.AAAAAQKljEW9<truncated>MABzdK
token_accessor       n/a
token_duration       768h
token_renewable      false
token_policies       ["default"]
identity_policies    []
policies             ["default"]

Create a token with a use-limit

You can create a token that can only be used a certain number of times. This is referred to as a token with a use-limit. To create a token that can only be used five times, run the following command:

$ vault token create -policy=default -use-limit=5

A token with a use-limit is useful when you know you have a process that requires a token a limited number of times.

1. A Vault development server is started by running vault server -dev in your terminal. ↩