Managing multiple Kube clusters (the easy way)

If you have ever tried to manage users on multiple Kubernetes clusters then you know the pain i am about to talk about.

The first thing you need to do after you bring up a cluster is define some privileges. You accomplish that by setting up roles and cluster roles.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  creationTimestamp: "2022-07-20T23:53:17Z"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: cluster-admin
  resourceVersion: "76"
  uid: 15d14062-0879-4232-9f42-51b79c0835f9
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
- nonResourceURLs:
  - '*'
  verbs:
  - '*'

After you have defined the rules in a role (outside of the scope of this article) then you need to create a clusterrolebinding or rolebinding that associated a particular user with a role or clusterrole. If this sounds overly complicated it's because it probably is.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  creationTimestamp: "2022-07-20T23:53:17Z"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: cluster-admin
  resourceVersion: "138"
  uid: 5ef726c1-01df-4ecd-8951-909698ef5472
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:masters

Oh yeah, and if forgot... You need to do this for every cluster you bring up.

I am here to tell you there is an easier way!

Easy Button

Using Portainer BE you can setup your users in the GUI.

You can then assign the users to groups, called teams.

Then you can associate the roles to groups per environment

You can now assign these same teams and roles to each envirnment. Kubernetes gets the user association in the backgrounds and the cluster roles are pushed down to each cluster. Now the only thing to do is to grab a kubeconfig file that is associated with that user.

All the clusters that user is associated with will be part of the kubeconfig and the kubeconfig points to a proxy that manages all of the calls. If a user is taken out of a team in portainer then those changes are immediate across clusters.

You can se how much easier it is to manage multiple cluster access this way. For more of a demo then checkout the video attached to this post.