At Focused Labs, we are fans of certificate-based auth. With certificates, you get all the nice safety of never sending a password over the wire, an easy way to revoke access, and more security than even with a generated password.

Now, you may want to use certificates to authenticate users or external services like CI with your Kubernetes cluster. You're in luck. There are many articles that look like they could help. You have probably found this, and seen this, and this, and maybe even this. But, after reading all that you were probably still left scratching your head. The following guide should help.

Our Goals

• Secure certificate provisioning
• Automatic CSR signing

By the end of this blog post you should be able to set up secure certificate authentication for users of your k8s Cluster.

Prerequisites

• You have a running cluster
• You have a basic working knowledge of the k8s API
• You have kubectl installed and are using a cluster admin context

Step One

Have the User Generate the CSR and Private Key

We believe very strongly in infrastructure as code. If it can be committed, it should be. So, our preference is using the tool cfssl because you can create a JSON config that's committable and shareable.

Let's configure cfssl and use envsubst to generate our private key and CSR.

{
  "CN": "$USERNAME",
  "names": [
      {
          "O": "$GROUP"
      }
  ],
  "key": {
    "algo": "ecdsa",
    "size": 256
  }
}

Your Common Name matters and so does your Organization.

"Kubernetes uses the Common Name to match the certificate with the username in the RoleBinding and uses the Organization to match the group in the RoleBinding."

This is hidden in their documentation here if you're curious.

An operator can share this file with a new user and have them run

export USERNAME=**ENTER USERNAME**
export GROUP=**ENTER GROUP**
envsubst < ../private_key_template.json | cfssl genkey -  | cfssljson -bare client

This will generate two files client-key.pem and client.csr.

That key file is secret - the user shouldn't share that with anyone.

Step Two

Sign the CSR and Generate a Cert for Auth

Have the new user send you the CSR, username, and group they used. Now, you can use Kubernetes to sign the CSR and get a client certificate for authentication.

With the client.csr create a CertificateSigningRequest

export USERNAME=**username*
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
  name: $USERNAME
spec:
  username: $USERNAME
  groups:
  - system:authenticated
  request: $(cat client.csr | base64 | tr -d '\n')
  usages:
  - digital signature
  - key encipherment
  - client auth
EOF

You can see the newly created CSR with kubectl get certificatesigningrequests

Now, approve the CSR and download a client certificate.

kubectl certificate approve $USERNAME
kubectl get csr $USERNAME -o jsonpath={.status.certificate} | base64 --decode > client.pem

You can send the new client.pem back to the new user.

Step Three

Create a Role and RoleBinding for the new user.

There is tons of documentation on this so we will keep the explanation brief. You will want to give rights to the newly authenticated user.

To do this, we create a Role that allows full access to a namespace with their username and a RoleBinding that gives this user access.¹

kubectl create namespace $USERNAME

cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: $USERNAME-namespace
rules:
- apiGroups: [""]
  resources: ["*"]
  verbs: ["*"]
EOF

cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: access-$USERNAME-namespace
  namespace: $USERNAME
subjects:
- kind: User
  name: $USERNAME
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: $USERNAME-namespace
  apiGroup: rbac.authorization.k8s.io
EOF

Step Four

Set up kubectl config to access the cluster

Now that the user can authenticate and has authorizations in the cluster, they need to set up their kubectl.

You (the operator) can send the new user

• client.pem
• ca.pem You'll need to get this from your cluster. It will be different depending on how you deployed Kubernetes.
• api endpoint This is the api for your cluster.

After the user receives everything from you they can use kubectl to modify their ~/.kube/config

kubectl config set-cluster internal --server=https://$API_ENDPOINT --certificate-authority=ca.pem --embed-certs=true
kubectl config set-credentials internal --client-certificate=client.pem --client-key=client-key.pem --embed-certs=true
kubectl config set-context prod --cluster=production --namespace=$USERNAME --user=$USERNAME

Finally, the user can use this new context

kubectl config use-context prod

They are all set.

Follow-ups

You can probably see that there is a ton of opportunity for automation. Most of the above can be done via a single script. And it's probably smart to create a CI or pipeline that automates and allows auditing of your cluster's access.

Expect part two to go in depth on the automation.

1. For production, we would recommend a more automated approach to what namespaces/access each user should have ↩