By: Scott Coulton, Microsoft Developer Advocate, CNCF Ambassador, and Docker Captain
One of the toughest pieces of the puzzle to solve when you are new to Kubernetes is ingress. Firstly, you have native ingress resource types, but you also need to use a controller. There are many third-party ingress controllers available to you, so what’s the difference between them?
Before going further, this is the second post in the series and will assume you have read the first post and have some understanding of Kubernetes services and namespaces. If you have not read the post you can find it here. If you are comfortable with services and namespaces, then you can continue reading.
Introduction to Ingress
Why do you need ingress in Kubernetes? You could just expose the service and specify type LoadBalancer. Right?
That would then speak to your cloud of choice and spin up a load balancer for you with a public IP address and your service would be available for consumption. Happy days!!!
Now imagine that you are running 10, or maybe 100 services exposed to the public. Each service would spin up its own load balancer and public IP address. You can see how this would very quickly sprawl and become unmanageable. Not to mention that it will be costing you money from your cloud provider for the extra load balancers and IP addresses.
Enter ingress. What ingress allows you to do is share a single public IP address and route your application via URLs or URI, commonly known as http routing. Once you have an ingress controller picked, you will be able to specify the URLs or URI that you need to route, and the controller will look after it for you.
Limitations to the ingress type
When using ingress there are a couple of limitations that you really need to be aware of.
The first is that ingress resources are not namespaced. Don’t get me wrong here, the controllers themselves can be namespaced with rbac controls. The definition of the actual ingress resource under Kind: “Ingress can’t not have rbac and is not namespaced.”
So, what does that mean?
This means that if you have multiple teams using the cluster, they will need to be wary of deployments involving ingress as if they have permissions to write their ingress; they will also have permission to change or modify someone else’s on the cluster.
This brings me to the second limitation of ingress; Kubernetes can have multiple of the same ingress types pointing to different resources. Now this can be of use to you, but this can also lead to traffic being routed to rouge services on your cluster that are not meant to be a part of that ingress rule. As mentioned earlier, this is something that you should be aware of when using ingress. Later on in the post, you’ll see some ways to make sure each ingress rule is unique.
How routing works
In the diagram below, you can see the flow of how requests are coming in from the internet, then hit the ingress controller, and are then routed to the service running on your cluster.
Now that you have a good understanding of the flow of traffic to your service, let’s look at how you can define the ingress resource.
For the examples, I am going to use the Nginx controller which is the easiest to get up and running. In the next section, you’ll find some other controllers to give you options for your cluster.
In the examples below, you’ll see that you can define your ingress rules via two methods: First with URI’s and the second with host headers.
Defining ingress via URI
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: production-uri
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: production.com
http:
paths:
- path: /users
backend:
serviceName: users-api
servicePort: 4200
- path: /payments
backend:
serviceName: payments-api
servicePort: 8080
In the above example, its been defined that there are two services on production.com: a users API and a payments API. You can see that if the traffic is going to production.com/users it will hit the service users-api.
Defining ingress via host header
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: my-production-v-host
spec:
rules:
- host: foo.production.com
http:
paths:
- backend:
serviceName: api-v1
servicePort: 80
- host: bar.production.com
http:
paths:
- backend:
serviceName: api-v2
servicePort: 80
In this example, its defining the routing via URL in the host header and your traffic will go to the service defined in the host header.
Choosing an ingress controller
Choosing an ingress controller will be a personal choice and will depend on your business use case. Here are some that have different options for configurations that are definitely worth a look.
Nginx is one of the easiest to get up and running. It does use the standard Kubernetes resource type, so you would need to implement something like open policy agent if you wanted to block multiple of the same entries of an ingress rule. Out of the box, Nginx gives you TLS/HTTPS support, load balancing, TCP/UDP, and interfaces to Prometheus for metrics which can come in handy.
Contour takes a completely different approach to ingress and uses a new ingress type via CRD’s (Kubernetes resource definitions). Remember earlier in the post there was mention of having multiple of the same ingress rule? Contour fixes this by only allowing one rule and also allows you to delegate your URI paths and parts of the URL. Additionally, it uses envoy as a reverse proxy. This is one that is worth your while researching.
- Traefik https://traefik.io/
Traefik is another popular choice for ingress and comes packed full of features. Traefik ships with a dashboard that allows you to dig into your ingress rules. There is also integrations with tracing (Jaeger, Open tracing and Zipkin), metrics (Prometheus or statsd), and TLS/HTTPS.
Depending on your business use case, you should definitely do some research on the open-source projects above for your ingress needs.
About: Scott Coulton
Scott Coulton is a Developer Advocate, CNCF Ambassador, and Docker Captain with 10 years of experience as a software engineer in the managed services and hosting space. He has extensive experience in architecture and rolling out distributed compute systems and network solutions for national and multinational companies with a wide variety of technologies, including Azure, Kubernetes, Puppet, Docker, Cisco, VMware, Microsoft, and Linux. His design strengths are in cloud computing, automation, and security space.
Top comments (1)
Good article, thanks! I've worked a fair bit with K8s from a local and dev cluster perspective, mostly setting up from scripts the infrastructure group at that role provided to the developer teams. I've worked with the nginx ingress a bit but have never had to actually figure out how to set one up, so this is a helpful overview.