Credits: Logo by traefik.io
Provide load balancing, SSL termination and name-based virtual hosting on a Kubernetes (k3s) cluster using Traefik ingress controller.
Note: The domain referenced in this post is MY_DOMAIN, please change accordingly. If you interested in a local-only work mode, you don’t have to pay for a new domain, just decide on a name and use it. For example, if your desired domain is homelab.com, replace MY_DOMAIN with homelab.
Prerequisites
- Setting Up a Raspberry Pi Cluster
- Install Rancher K3s on Raspberry Pi Cluster
- Installing Helm, Kubernetes Package Manager
- Install Certificate Manager Controller in Kubernetes
What is an Ingress Controller?
It is an API object that manages external access to a deployed service in a Kubernetes cluster, typically via HTTP/S. It provides load balancing, SSL termination and name-based virtual hosting.
Why would I want to use an Ingress Controller?
These are some of the immediate benefits using an ingress controller:
-
Simplify the way internal services interact with each other and re-route when required by changing only the Traefik routing rule.
Note: Instead of relying on an IP address that might change, simply define a hosted name e.g.
serviceA.domain.com
to addressserviceA
and use that on other internal services and/or from outside the cluster when in need to callserviceA
.
Isn'tserviceA.domain.com
easier to remember than192.169.200.xxx
? Allows HTTPS traffic from outside the Kubernetes cluster while terminating encryption and allowing HTTP traffic between services within the cluster
Load balance traffic between services hosted outside the Kubernetes cluster
Note: Traefik is using custom cluster resource definitions. These CRDs gets installed via Traefik 2 Helm Chart. For CRDs schema click here.
K8s Controller
Traefik is a Kubernetes controller that manage the access to cluster services by supporting the Ingress specification. It receives requests on behalf of your system and finds out which components are responsible for handling them.
Prepare
We want Kubernetes to create the Traefik pod on the master node. In order to do that, we'll have to label that node and use nodeSelector
attribute when installing Traefik Helm chart.
-
Get all nodes names and labels
kubectl get nodes --show-labels
-
Label
kmaster
node withnode-type=master
kubectl label nodes kmaster node-type=master
Note: To remove a label, use the same command with dash after the label name e.g.
kubectl label nodes kmaster node-type-
." -
Verify that label had been created successfully
kubectl get nodes --show-labels | grep node-type
Install
-
Create a
traefik
namespace
kubectl create namespace traefik
-
Add the
containous
Helm repository hosting the Traefik charts metadata
helm repo add traefik https://containous.github.io/traefik-helm-chart
-
Update local Helm chart repository cache
helm repo update
-
Search for latest
traefik/traefik
official Helm chart version
helm search repo traefik # NAME CHART VERSION APP VERSION # traefik/traefik 9.1.1 2.2.8
-
Install the Traefik Helm chart using the version from previous step
helm upgrade --install traefik \ --namespace traefik \ --set dashboard.enabled=true \ --set rbac.enabled=true \ --set nodeSelector.node-type=master \ --set="additionalArguments={--api.dashboard=true,--log.level=INFO,--providers.kubernetesingress.ingressclass=traefik-internal,--serversTransport.insecureSkipVerify=true}" \ traefik/traefik \ --version 9.1.1
-
Verify installation
# Make sure all traefik deployed pods are running kubectl get pods --namespace traefik # Make sure custom resources *.traefik.containo.us were created successfully kubectl get crd | grep traefik
Uninstall
-
Remove
traefik
from the cluster
helm uninstall traefik --namespace traefik
-
Clear the
traefik
namespace
kubectl delete namespaces traefik
Traefik Dashboard
Dashboard is installed but disabled by default for security reasons. We will want to avoid using the kubectl proxy-forward
option and allow the dashboard via HTTPS with proper TLS/Cert.
Certificate
We will create a certificate using cert-manager
to allow accessing the Traefik dashboard via the hosted name traefik.MY_DOMAIN.com
within our home network. Create a self signed certificate as described in here under traefik
namespace.
Verify that a TLS secret had been created for the certificate:
kubectl get secret MY_DOMAIN-com-cert-secret --namespace traefik
Authentication
We will create a user / password basic authentication, please read here if you wish to use a different method.
-
Generate a flat-file that stores a username and password for basic authentication
bash <<'EOF' # Change these credentials to your own export TRAEFIK_UI_USER=admin export TRAEFIK_UI_PASS=dashboard export DESTINATION_FOLDER=${HOME}/temp/traefik-ui-creds # Backup credentials to local files (in case you'll forget them later on) mkdir -p ${DESTINATION_FOLDER} echo $TRAEFIK_UI_USER >> ${DESTINATION_FOLDER}/traefik-ui-user.txt echo $TRAEFIK_UI_PASS >> ${DESTINATION_FOLDER}/traefik-ui-pass.txt htpasswd -Bbn ${TRAEFIK_UI_USER} ${TRAEFIK_UI_PASS} \ > ${DESTINATION_FOLDER}/htpasswd unset TRAEFIK_UI_USER TRAEFIK_UI_PASS DESTINATION_FOLDER EOF
-
Create a Kubernetes secret based on the basic authentication file
kubectl create secret generic traefik-dashboard-auth-secret \ --from-file=$HOME/temp/traefik-ui-creds/htpasswd \ --namespace traefik
Ingress
-
Create an
IngressRoute
for accessing the dashboard, make sure to replaceMY_DOMAIN
with your domain name
cat <<EOF | kubectl apply -f - apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: traefik-dashboard namespace: traefik spec: entryPoints: - websecure routes: - kind: Rule match: Host(\`traefik.MY_DOMAIN.com\`) && (PathPrefix(\`/api\`) || PathPrefix(\`/dashboard\`)) services: - name: api@internal kind: TraefikService middlewares: - name: traefik-dashboard-auth # Referencing the BasicAuth middleware namespace: traefik tls: secretName: MY_DOMAIN-com-cert-secret --- apiVersion: traefik.containo.us/v1alpha1 kind: Middleware metadata: name: traefik-dashboard-auth namespace: traefik spec: basicAuth: secret: traefik-dashboard-auth-secret EOF
-
Check that resources were created successfully
# IngressRoute kubectl describe ingressroute traefik-dashboard --namespace traefik # Middleware kubectl describe middleware traefik-dashboard-auth --namespace traefik
-
Check there are no error logs
kubectl logs -f $(kubectl get pods --namespace traefik | grep "^traefik" | awk '{print $1}') --namespace traefik
-
Set a hosted name
traefik.MY_DOMAIN.com
on a client machine (laptop/desktop)
# Edit the file using `sudo /etc/hosts` # Append manualy to the existing k3s hosted name 111.222.333.444 kmaster, traefik.MY_DOMAIN.com # Alternatively, add a new hosted name entry with a one-liner echo -e "111.222.333.444\ttraefik.MY_DOMAIN.com" | sudo tee -a /etc/hosts
Note: Replace
111.222.333.444
with thek3s
master node IP address andMY_DOMAIN
with your domain name. -
Open browser at https://traefik.MY_DOMAIN.com/dashboard/
Note: If you encounter an untrusted certificate warning, please follow the cert-manager post Trust section for instructions on how to trust the
traefik
certificate.
Demo Application
We will deploy an example whoami
application that returns basic information about the client issuing the request. It'll allow us to check that Traefik was deployed successfully by addressing our demo application using whoami.MY_DOMAIN.com
.
Prepare
Before deploying our application to the Kubernetes cluster, we will have to define which node(s) are eligible of assigning pods for it.
We will use the node-type=master
label created at the Prepare
step of the Traefik installation. To assign the whoami
pod(s) to the kmaster
node, we will use the nodeSelector
attribute on a Kubernetes resource (Deployment
/ Pod
) with that label.
Install
-
Create a
playground
namespace
kubectl create namespace playground
-
Create Kubernetes
Deployment
andService
with x2 running instances of thewhoami
application
cat <<EOF | kubectl apply -f - apiVersion: apps/v1 kind: Deployment metadata: name: whoami namespace: playground labels: app: containous name: whoami spec: replicas: 2 selector: matchLabels: app: containous task: whoami template: metadata: labels: app: containous task: whoami spec: nodeSelector: node-type: "master" containers: - name: containouswhoami image: containous/whoami ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: whoami namespace: playground spec: ports: - name: http port: 80 selector: app: containous task: whoami EOF
Note: Please refer to the official docs to read more about Kubernetes
Deployment
orService
. -
Verify both
Deployment
andService
were created successfully
# Deployment should indicate 2/2 running pods kubectl get deployments -n playground # Service should indicate a cluster IP on port 80 without an external IP kubectl get services -n playground # Check that there are x2 running pods on kmaster node kubectl describe pods whoami -n playground | grep "Status:"
Uninstall
-
Follow the installation instructions for
Deployment
,Service
,IngressRoute
,Middleware
and execute both scripts while replacing thekubectl
action:
# Instead of using apply cat <<EOF | kubectl apply -f - ... EOF # Replace with delete cat <<EOF | kubectl delete -f - ... EOF
-
Clear the
playground
namespace
kubectl delete namespaces playground
Certificate
We will have to create a certificate using cert-manager
to allow accessing the whoami
application using the hosted name whoami.MY_DOMAIN.com
within our home network. Create a self signed certificate as described in here under playground
namespace.
Verify that a TLS secret had been created for the certificate:
kubectl get secret MY_DOMAIN-com-cert-secret --namespace playground
Ingress
-
Create ingress entry points to allow secure communication using
HTTPS
and allowingHTTP
withHTTPS
redirect middleware (make sure to replaceMY_DOMAIN
with your domain name)
cat <<EOF | kubectl apply -f - apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: whoami-https namespace: playground spec: entryPoints: - websecure routes: - match: Host(\`whoami.MY_DOMAIN.com\`) kind: Rule priority: 10 services: - name: whoami port: 80 tls: secretName: MY_DOMAIN-com-cert-secret --- apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: whoami namespace: playground spec: entryPoints: - web routes: - match: Host(\`whoami.MY_DOMAIN.com\`) kind: Rule priority: 10 services: # This IngressRoute will be never called due to the redirect middleware - name: whoami port: 80 middlewares: - name: https-redirect --- apiVersion: traefik.containo.us/v1alpha1 kind: Middleware metadata: name: https-redirect namespace: playground spec: redirectScheme: scheme: https permanent: true EOF
-
Verify success status of HTTP routers and middlewares using the traefik dashboard
-
Set a hosted name
whoami.MY_DOMAIN.com
on a client machine (laptop/desktop)
# Edit the file using `sudo /etc/hosts` # Append manualy to the existing k3s hosted name 111.222.333.444 kmaster, traefik.MY_DOMAIN.com, whoami.MY_DOMAIN.com # Alternatively, add a new hosted name entry with a one-liner echo -e "111.222.333.444\twhoami.MY_DOMAIN.com" | sudo tee -a /etc/hosts
Note: Replace
111.222.333.444
with thek3s
master node IP address andMY_DOMAIN
with your domain name. -
Open a web-browser at
https://whoami.MY_DOMAIN.com
and check that you can get a response from the service(s)Note: Open
http://whoami.MY_DOMAIN.com
to check theHTTP
->HTTPS
redirect.
Summary
Congratulations on completing a major part of deploying an ingress controller on your Kubernetes cluster ! 💪
What now? Now that you are familiar with how to use Traefik as an ingress controller, you can simplify the inter-services communication within the cluster and expose applications APIs / Web-UI and such via HTTPS
in no time.
Please leave your comment, suggestion or any other input you think is relevant to this post in the discussion below.
Like this post?
You can find more by:
Checking out my blog: https://blog.zachinachshon.com
Following me on twitter: @zachinachshon
Thanks for reading! ❤️
Top comments (0)