Creating the Admission Controller
Before kube-apiserver persists the object and later be scheduled to a node,
the Admission Controller can validate and mutate the object.
The Mutation Admission Controller is going to mutate the pods that have the the
annotation diy-service-mesh: true.
Let's dig into how the Admission Controller works.
Admission Controller Flow
- kube-controller or kubectl sends a request to the kube-apiserver to create a pod.
- The kube-apiserver sends the request to the Admission Controller. In this case the proxy-injector.
- The proxy-injector returns the mutated patch to the kube-apiserver.
- Kube-apiserver persists the object in the etcd if the object is valid.
- Kube-scheduler will schedule the pod to a node.
- Kube-scheduler returns an available node to the kube-apiserver or an error if the pod can't be scheduled.
- The kube-apiserver will store the object in the etcd with the selected node.
- The kubelet in the selected node will create the pod in the container runtime.
Admission Controller Code
Full code of the Admission Controller: injector
The mutate function processes the AdmissionReview object and returns the AdmissionResponse object with the mutated patch.
func mutate(ar *admissionv1.AdmissionReview) *admissionv1.AdmissionResponse {
req := ar.Request
// Ignore all requests other than pod creation.
if req.Operation != admissionv1.Create || req.Kind.Kind != "Pod" {
return &admissionv1.AdmissionResponse{
UID: req.UID,
Allowed: true,
}
}
var pod v1.Pod
// Unmarshal the raw object to the pod.
if err := json.Unmarshal(req.Object.Raw, &pod); err != nil {
return &admissionv1.AdmissionResponse{
UID: req.UID,
Result: &metav1.Status{
Message: err.Error(),
},
}
}
// Check if the pod contains the inject annotation.
if v, ok := pod.Annotations["diy-service-mesh/inject"]; !ok || strings.ToLower(v) != "true" {
return &admissionv1.AdmissionResponse{
UID: req.UID,
Allowed: true,
}
}
// Add the initContainer to the pod.
pod.Spec.InitContainers = append(pod.Spec.InitContainers, v1.Container{
Name: "proxy-init",
Image: os.Getenv("IMAGE_TO_DEPLOY_PROXY_INIT"),
ImagePullPolicy: v1.PullAlways,
SecurityContext: &v1.SecurityContext{
Capabilities: &v1.Capabilities{
Add: []v1.Capability{"NET_ADMIN", "NET_RAW"},
Drop: []v1.Capability{"ALL"},
},
},
})
// Add the sidecar container to the pod.
pod.Spec.Containers = append(pod.Spec.Containers, v1.Container{
Name: "proxy",
Image: os.Getenv("IMAGE_TO_DEPLOY_PROXY"),
ImagePullPolicy: v1.PullAlways,
SecurityContext: &v1.SecurityContext{
RunAsUser: func(i int64) *int64 { return &i }(1337),
RunAsNonRoot: func(b bool) *bool { return &b }(true),
},
})
patch := []map[string]any{
{
"op": "replace",
"path": "/spec/initContainers",
"value": pod.Spec.InitContainers,
},
{
"op": "replace",
"path": "/spec/containers",
"value": pod.Spec.Containers,
},
}
podBytes, err := json.Marshal(patch)
if err != nil {
return &admissionv1.AdmissionResponse{
UID: req.UID,
Result: &metav1.Status{
Message: err.Error(),
},
}
}
patchType := admissionv1.PatchTypeJSONPatch
return &admissionv1.AdmissionResponse{
UID: req.UID,
Allowed: true,
AuditAnnotations: map[string]string{
"proxy-injected": "true",
},
Patch: podBytes,
PatchType: &patchType,
}
}
IMAGE_TO_DEPLOY_PROXY_INIT and IMAGE_TO_DEPLOY_PROXY are environment variables that tilt will update with the last proxy-init and proxy image respectively.
For complex patch use thi library: https://github.com/evanphx/json-patch
Deploying the Admission Controller
MutatingWebhookConfiguration tells the kube-apiserver to send the pod creation requests to the injector.
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: service-mesh-injector-webhook
webhooks:
- name: service-mesh-injector.service-mesh.svc
clientConfig:
service:
name: service-mesh-injector
namespace: service-mesh
path: "/inject"
rules:
- operations: ["CREATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
admissionReviewVersions: ["v1"]
sideEffects: None
timeoutSeconds: 5
Some important points:
-
caBundlein theclientConfigis missing. This is a necessary field because the kube-apiserver only calls the webhook if the certificate is valid. - Two jobs,
injector-admission-createinjector-admission-patchare going generate the certificates and patch theMutatingWebhookConfigurationwith thecaBundle. - The
rulesoptions allow to filter the objects that are going to be sent to the injector.
The file injector.yaml contains the nesesary resources
including the Service Account, Role, RoleBinding, ClusterRole, ClusterRoleBinding,
Service, Deployment and the Job to generate the certificates.
Testing the Admission Controller
Let's modify the http-client and http-server deployments to add the annotation diy-service-mesh/inject: "true".
spec:
replicas: 1
selector:
matchLabels:
app: http-client
template:
metadata:
labels:
app: http-client
annotations:
diy-service-mesh/inject: "true"
spec:
Important: the annotation needs to be added to the pod template and not to the deployment.
Top comments (0)