Keycloak, an open-source identity and access management solution, provides robust authentication and authorization services for modern applications. However, configuring Keycloak instances manually can be tedious and error-prone. In this blog post, we'll explore how to simplify Keycloak configuration using Terraform and Terragrunt, enabling infrastructure as code (IaC) practices for managing Keycloak realms, clients, users, and more.
Why Terraform and Terragrunt?
Terraform is an open-source tool that lets you manage your infrastructure as code. With Terraform, you can define your infrastructure in simple configuration files, and Terraform will automatically create, manage, and update your infrastructure according to those configurations.
Terragrunt is a thin wrapper for Terraform that provides extra tools for keeping your Terraform configurations DRY, managing remote state, and working with multiple Terraform modules. It helps you maintain clean, DRY, and repeatable Terraform code by providing extra functionality and best practices.
By leveraging both, we can have easy:
- Infrastructure as Code (IaC): By defining Keycloak configuration as code, we can version control, review changes, and ensure consistency across environments.
- Modularization: Modularize our Keycloak configuration, making it easier to manage complex setups.
- State Management: Manage the state of our infrastructure, preventing configuration drift and ensuring that our infrastructure remains in the desired state.
Let us go!
Install Terraform and Terragrunt
Make sure you have Terraform and Terragrunt installed on your machine. You can find installation instructions on the official Terraform and Terragrunt documentation
- Terraform: https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli
- Terragrunt: https://terragrunt.gruntwork.io/docs/getting-started/install/
Project Structure
Creating a one-size-fits-all structure for a Terraform project can be challenging because it largely depends on the specific requirements of each project. Below is the structure I've found most suitable for organizing Keycloak components, concepts, and setup.
.
├── README.md <- The project overview
├── .tool-versions <- Used tools versions (managed by asdf. see https://asdf-vm.com)
├── README.md <- The project overview
├── modules <- Terraform modules
| └── common
│ ├── provider.tf
│ └── variables.tf
│ └── output.tf
│ └── main.tf
│ └── README.md
│ └── docs/
| └── clients
│ ├── provider.tf
│ └── variables.tf
│ └── output.tf
│ └── main.tf
│ └── README.md
│ └── docs/
| └── my-realm
│ ├── provider.tf
│ └── variables.tf
│ └── output.tf
│ └── main.tf
│ └── README.md
│ └── docs/
| └── other
│ ├── provider.tf
│ └── variables.tf
│ └── output.tf
│ └── main.tf
│ └── README.md
│ └── docs/
└── how-to <- Documentation
└── stage <- Terraform for environment stage
├── .terraform.lock.hcl <- Terraform lock file
└── terragrunt.hcl <- Terragrunt file
└── env.yaml <- environment related variables
└── main.tf <- environment modules
└── prod <- Terraform for environment prod
├── .terraform.lock.hcl <- Terraform lock file
└── terragrunt.hcl <- Terragrunt file
└── env.yaml <- environment related variables
└── main.tf <- environment modules
└── local <- Terraform for environment local
├── .terraform.lock.hcl <- Terraform lock file
└── terragrunt.hcl <- Terragrunt file
└── env.yaml <- environment related variables
└── main.tf <- environment modules
In this project structure, I've included a modules directory containing a set of modules shared across all environments. Each module includes a main.tf
file encapsulating the module's resources, along with input.tf
, output.tf
, and variable.tf
files for easy configuration across different environments.
Common Module
Let's assume that in the common
module, we configure realm events by using the jboss-logging
event listener with some non-default configurations. Below is an example of how the main.tf
file may look:
resource "keycloak_realm_events" "realm_events" {
realm_id = var.realm_id
events_enabled = true
events_expiration = 1800
admin_events_enabled = true
admin_events_details_enabled = true
]
events_listeners = [
"jboss-logging"
]
}
To include the realm_id
variable, it must be defined in the variables.tf
file as shown below:
variable "realm_id" {
description = "Realm ID"
type = string
}
And we must configure the used providers in our module. In this case, the provider.tf
will look like as:
terraform {
required_providers {
keycloak = {
source = "mrparkers/keycloak"
}
}
}
Master Realm Module
In the realm-master
module, the main.tf
file should reference the common module. Here is an example of how the main.tf
file may look:
data "keycloak_realm" "master" {
realm = "master"
}
module "realm-master" {
source = "../../modules/common"
realm_id = data.keycloak_realm.master.id
}
Similarly, the provider.tf
file should be configured as follows:
terraform {
required_providers {
keycloak = {
source = "mrparkers/keycloak"
}
}
}
Local environment
In the main.tf
file, we need to define the master realm in order to reference the realm-master
module in our project.
# Define master realm
module "realm-master" {
source = "../modules/realm-master"
}
Within each environment (e.g., prod
, stage
and local
), there's an env.yaml
file containing all the environment-specific variables.
For example, the env.yaml
file for the local environment may look like this:
---
environment: local
url: http://localhost:8080/keycloak
And of course, don't forget to include the terragrunt-local.hcl
file, which should be defined in the parent module.
include "root" {
path = find_in_parent_folders("terragrunt-local.hcl")
}
Terragrunt configuration
As mentioned earlier, Terragrunt is highly beneficial for keeping your configuration Don't Repeat Yourself (DRY). For example, the terragrunt.hcl
file may look like this:
# Generates the backend for all modules.
remote_state {
backend = "s3"
config = {
encrypt = true
key = "keycloak/${path_relative_to_include()}/terraform.tfstate"
region = "<AWS REGION>"
bucket = "terraform-states"
dynamodb_table = "terraform-lock"
}
}
# Read the local "env.yaml" in every environment.
locals {
vars = yamldecode(file("${path_relative_to_include()}/env.yaml"))
environment = local.vars.environment
url = local.vars.url
}
# Generate the "provider.tf" file for every module.
generate "provider" {
path = "provider.tf"
if_exists = "overwrite"
contents = <<EOF
terraform {
required_providers {
keycloak = {
source = "mrparkers/keycloak"
version = "4.4.0"
}
http = {
source = "hashicorp/http"
version = "3.2.1"
}
}
}
data "http" "config" {
url = "<INTERNAL CONFIGURATION URL>"
}
provider "keycloak" {
client_id = jsondecode(data.http.config.response_body).terraform-client-id
client_secret = jsondecode(data.http.config.response_body).terraform-client-secret
url = "${local.url}"
}
EOF
}
# Generate the "backend.tf" file for every module.
generate "backend" {
path = "backend.tf"
if_exists = "overwrite"
contents = <<EOF
terraform {
backend "s3" {}
}
EOF
}
If you're familiar with Terragrunt, you'll find the terragrunt.hcl
file above quite familiar, except for the http config
part, which I'll describe later in section.
In remote_state
, we use AWS S3 to store the state of our environment configurations. Additionally, we generate the backend
and provider
for every environment.
The locals
block is mainly used to read the env.yaml
file and assign the variables to local variables.
Lastly, the http config
is responsible for making an HTTP call to wherever you securely store your environment configurations (at least the Terraform client ID and secret) to pass them to the keycloak
provider.
The terragrunt-local.hcl
file is similar to the terragrunt.hcl
file, except for the remote state configuration, which isn't necessary in this case. The local
file is primarily for testing your configuration on a local Keycloak cluster setup.
The Docker Compose
To run our Terraform configurations, we require a Keycloak cluster setup. In below, we use Docker Compose to start a Keycloak cluster, enabling us to run our local Terraform configurations seamlessly against it.
version: '3'
volumes:
postgres_data:
driver: local
services:
postgres:
image: postgres
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
POSTGRES_DB: keycloak
POSTGRES_USER: keycloak
POSTGRES_PASSWORD: password
ports:
- "5432:5432"
keycloak:
image: quay.io/keycloak/keycloak:23.0.6
environment:
KC_DB_USERNAME: keycloak
KC_DB_PASSWORD: password
KC_DB_URL_HOST: postgres
KC_DB: postgres
KC_DB_SCHEMA: public
KC_HTTP_RELATIVE_PATH: /keycloak
KC_HOSTNAME_ADMIN: 127.0.0.1
KC_HOSTNAME: localhost
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
command:
- start-dev
ports:
- "8080:8080"
- "8787:8787"
depends_on:
- postgres
config_keycloak:
image: ubuntu
volumes:
- ./keycloak-docker-config.sh:/opt/keycloak-docker-config.sh
command: ./opt/keycloak-docker-config.sh
depends_on:
- keycloak
The keycloak-docker-config.sh
script is primarily used to configure a Terraform client with admin
privileges, which Terraform will use during its operations.
#!/bin/bash
apt update -y && apt -y install jq curl
until $(curl --output /dev/null --silent --head --fail http://keycloak:8080/keycloak); do
printf '.'
sleep 5
done
# Get access token
TOKEN=$( \
curl -X POST \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "client_id=admin-cli" \
-d "username=admin" \
-d "password=admin" \
-d "grant_type=password" \
"http://keycloak:8080/keycloak/realms/master/protocol/openid-connect/token" | jq -r '.access_token')
# Create Terraform client (terraform/terraform)
curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${TOKEN}" \
-d '{"clientId": "terraform", "name": "terraform", "enabled": true, "publicClient": false, "secret": "terraform", "serviceAccountsEnabled": true}' \
"http://keycloak:8080/keycloak/admin/realms/master/clients"
# Get the Terraform service account user ID
USER_ID=$( \
curl -X GET \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${TOKEN}" \
"http://keycloak:8080/keycloak/admin/realms/master/users?username=service-account-terraform" | jq -r '.[0].id')
# Get the admin role ID
ROLE_ID=$( \
curl -X GET \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${TOKEN}" \
"http://keycloak:8080/keycloak/admin/realms/master/roles" | jq -r '.[] | select(.name == "admin") | .id')
# Add the admin role to the Terraform service account user
curl -kv -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${TOKEN}" \
-d '[{"id":"'"${ROLE_ID}"'", "name":"admin"}]' \
"http://keycloak:8080/keycloak/admin/realms/master/users/$USER_ID/role-mappings/realm"
To run it, open the terminal
and run the below docker compose command
docker-compose up --build
After the docker-compose containers are up and running, navigate to http://localhost:8080/keycloak
and log in using admin/admin
as credentials. Ensure that the Terraform client is configured within the master realm.
Now it's time to run your Terraform local configurations. Open your terminal and execute the following commands:
# Navigate to the local environment
$ cd local
# Ensure that Terraform-related files, including the auto-generated backend.tf and provider.tf, are removed
$ rm -r backend.tf provider.tf terraform.tfstate terraform.tfstate.backup .terraform.lock.hcl .terraform
# Initialize Terraform to create all necessary files
$ terragrunt init --terragrunt-config terragrunt-local.hcl
# Apply the Terraform configurations
$ terragrunt apply --terragrunt-config terragrunt-local.hcl
Now, open your web browser and go to http://localhost:8080/keycloak
. Log in using your admin credentials and make sure your configurations are properly set up there!
I created a demo project on GitHub keycloak-terraform-demo
Conclusion
Organizing and managing Keycloak configurations with Terraform can greatly streamline your development process. By following the structure and steps outlined in this guide, you can efficiently set up and maintain your Keycloak environments, ensuring consistency and scalability across your projects. If you have any questions or suggestions, feel free to leave them in the comments below.
I hope you find it useful!
Top comments (5)
Nice!! How to update master realm for example the policies. That means the attributes of master realm -> Authentication -> Policies should be updated.
Could you pls. suggest.
get the "realm master"
data "keycloak_realm" "master" {
realm = "master"
}
Then how to update ?
Doing the below throws duplicate exception
resource "keycloak_realm" "master_realm" {
realm = data.keycloak_realm.master.id
otp_policy {
digits = 8
}
}
Thank you.
Hi Mohammed, do you have a git on this topic?
Hi @effixion, unfortunately no but I can make one if you still needed it.
would be very helpful for me to have one!
Hi! I created the demo project here github.com/mohammedalics/keycloak-.... Sorry for being late on this update. I hope you find it useful.