K8s is overkill for your workload

I hate to be the bearer of bad news, but since you’re reading this, I guess that, most likely, you’re running Kubernetes in production. Sorry to break it to you: chances are - you’re using the wrong tool for the job.

In the IT industry, few other products in recent years have become so well known or become such an industry buzzword as Kubernetes. In a relatively short time, Kubernetes has become how organizations, from small startups to internet-sized enterprises, run their container workloads to organize, connect, and scale applications on demand.

What is Kubernetes?

If you haven’t heard of Kubernetes before, let me summarize. Kubernetes, known as k8s, is a widely recognized and extensively used platform in the IT industry. It allows organizations to manage and scale their container workloads by providing comprehensive tools and features. Kubernetes enables seamless organization, connectivity, and scalability of applications, making it a popular choice for businesses of all sizes, from small startups to large enterprises.

K8s has its roots in 2015 as a Google project and was built upon Google’s cluster management system known as Borg. Google worked with the Linux Foundation to kickstart the Cloud Native Foundation (CNCF), a non-profit organization that nourishes and grows open-source software projects. It offered Kubernetes to the CNCF as its initial seed project.

Why Kubernetes has become so popular

Kubernetes has gained immense popularity recently thanks to its ability to manage and scale container workloads effectively, and its comprehensive set of tools and features allows organizations to organize, connect seamlessly, and scale applications. K8s has become the preferred choice for businesses of all sizes. Kubernetes provides a resilient and highly available infrastructure, automates tasks, ensures efficient resource utilization, and simplifies application deployment and management. Its flexibility, portability, and compatibility with various cloud providers have further contributed to its widespread adoption.

Kubernetes’ excellent cloud compatibility is also one of its most substantial advantages. All three major public cloud vendors and many private cloud vendors have implemented Kubernetes in some form, creating abstraction layers to reduce the complexity for the end users.

But this is also the crux of K8s - complexity

DevSecOps and Kubernetes

Taking one step back for a second, there is another factor we need to keep in mind. Alongside the rise of Kubernetes, a movement has risen in the IT world: bringing developers (Dev), Security (Sec), and Operations (Ops) together. Although DevSecOps is a topic for a post on its own, I argue that without its rise of DevSecOps, Kubernetes would not have been as popular as it has become in that short time.

Allowing developers to write (relatively) simple application declarations and then have a (somewhat) easy way to throw them up in a cluster and have the application up and running quickly is a massive plus for many businesses! If you have a company pushing for SRE and app teams to work together to build something great, this can be a potent combination.

However - this perfect world is often not the reality we live in…

Kubernetes might not be the right tool for the job

Imagine you’re going to build a house. If you want to do the job well, you will need several tools. A power drill will undoubtedly help you a long way, but you will get to a point where you probably will need other tools too: you will probably need a hammer, a nail gun, a bubble level, a saw - you name it, and this is also the case with Kubernetes.

Kubernetes is **great**. But if you’re only running a few containers or microservices here and there, using Kubernetes for that work is like bringing a battleship to a water balloon fight - it’s overkill. Even if you’re running larger workloads, 10 or 20 containers, you could still have a better chance using other readily available container runtime environments or something completely different, such as serverless functions.

The issue we keep encountering across hundreds of businesses is that Kubernetes is not just Kubernetes. If you’re going to run a Kubernetes cluster ********effectively********, you need so much more. You most likely need a way to deploy to K8s in some CI/CD pipeline, then have something keep track of your running pods and versioning, such as Argo. It would be best if you had a way to keep track of secrets, logs, authentication, some service mesh, and so much more. Using Kubernetes in production is an iceberg. It's a vast iceberg; like real icebergs, you only typically see the top 10%.

Fig 1: An example Kubernetes Iceberg

And here is the crux of the problem - most businesses are not set up to handle this sort of iceberg! We don’t have enough engineers with the infrastructure experience to deal with the intricacies that come with such a complex issue that is K8s, and what we keep seeing time and time again happen to businesses is developers being forced to do infrastructure work and hating it.

This isn’t always the case. Many developers enjoy working with K8s, and doing infrastructure work is often a new and exciting challenge for them, but we see burnout rates explode for those who don’t enjoy it and are being “forced” to do that sort of work.

Developer satisfaction is not the only thing to consider here. What we see time and time again is a lack of resources on the operations/platform or security sides, and you struggle to maintain the cluster. Kubernetes has a reasonably frequent release schedule and regularly releases breaking changes, and this is just talking about the cluster itself. It becomes much more complicated when you start looking at maintaining the nodes and their operating systems, plus all the Kubernetes dependencies.

And the fact is - chances are, you’re not developing an internet-scale application. You’re bringing a battleship to a water balloon fight.

Managed Kubernetes

One way to relieve some of this pain is to run managed Kubernetes in some form, typically from a public cloud provider. This is undoubtedly an excellent way to reduce significantly the workload your team needs to do to keep up with the platform. Google Kubernetes Engine (GKE), Amazon Elastic Kubernetes Service (EKS), and Azure Kubernetes Service (AKS) are all excellent ways to avoid having to maintain the K8s nodes themselves. However, you’re still not eliminating the issue of Kubernetes dependencies, such as tools and services that typically are the leading resource thieves.

The alternatives

So, what should you consider instead? First, let’s clarify: we don’t want to return to Virtual Machines. We want to move forward to newer and better things. So what are they?

If you still want to keep running containers with minimal changes to your workflow, consider running in a completely managed container environment such as Amazon Elastic Container Service (ECS) or, if you have a small app or limited services, Azure Container Apps. ECS is an excellent production and internet-scale equivalent to K8s/EKS, but without the need to maintain a cluster - everything is done by using managed Services in AWS. Here, you still use the same containers you have been using and have a very minimal declaration and configuration setup, then let Amazon do the rest for you.

However, if you want even less maintenance and are prepared for an application rewrite, cloud functions alongside an event-driven architecture can do wonders for your teams with less complexity and less work for everyone involved. For some, it might be slightly more expensive per compute hour, but based on experience, you’ll quickly make up that difference in engineering hours, which are typically significantly more expensive.

Fig 2: Example of an event-driven architecture (Source: Amazon Event-Driven Architecture)

The future?

Managed services are usually the way to go for a lot of teams. Most organizations simply don’t have the resources to build a well-designed and well-built production environment without cutting back on developer productivity. By combining the usage of well-maintained managed services and considering using a different tool for the job, you can significantly increase developer satisfaction and productivity - both of which are often in short supply. Adopting these managed services is also often not something that takes a lot of time or is hard to do, and it can be the quick win many teams are looking for.

As stated before - if you’re looking for long-term changes or starting from scratch - an event-based infrastructure will set you up for the future and allow you to scale quickly and easily.

But what do you think? Should everyone adopt Kubernetes, or do you know of good alternatives businesses should consider?