Deploying Kafka On Kubernetes NOW Easier With Confluent Operator

The Confluent Operator

The widespread use of Kubernetes by the innumerable number of platforms in the company gave rise to Confluent Operator – says Neha Narkhede, Co-founder and CTO at Confluent – “the best way to deploy Apache Kafka and Confluent Platform on Kubernetes“. Alex Woodie rightly noticed that “many organizations have attempted to wire Kafka to run on the Kubernetes cluster manager, but it has been a challenging technical feat that Confluent has succeeded at!”

“Enabling everyone to run Apache Kafka® on Kubernetes is an important part of our mission to put a streaming platform at the heart of every company. This is why we look forward to releasing an implementation of the Kubernetes Operator API for automated provisioning, management, and operations of Kafka on Kubernetes” says Neha Narkhede.

“This includes a rollout of pre-built Docker images for the entire Confluent Platform, Kubernetes deployment templates, a reference architecture that enables users to deploy Kafka and Confluent Platform on Kubernetes and an implementation of the Kubernetes Operator API for automated provisioning, management, and operations of Apache Kafka on Kubernetes” – adds Confluent’s CTO.

Kafka + Kubernetes expertise = The Confluent Operator

Rohit Bakhshi, the author of the article “Getting Started with Apache Kafka and Kubernetes” best sums up this solution:

“Our goal is to make streaming data ubiquitous: Kubernetes lets you run your apps and services anywhere, Kafka enables you to make your data accessible instantaneously, anywhere”.

Despite the fact that the link between Apache Kafka on Kubernetes may seem like a ‘match in big data heaven’, the devil is in the details. What were the biggest obstacles to overcome? Read more.

Why Should Your Deploy Kafka On Kubernetes With Confluent Operator?

Neha Narkhede explains that “the Confluent Operator directly addresses the challenges of running Apache Kafka on Kubernetes, and will offer the following features across all Confluent Platform components”:

1. Automated Provisioning
2. Cluster Management and Operations
3. Resiliency
4. Monitoring Read more

Getting Started – How To Easily Run Apache Kafka on Kubernetes

Confluent provides two resources to install Kafka on Kubernetes:

1. Helm Charts – a starting point for deployment of Confluent Platform components

2. Recommendations for Deploying Apache Kafka on Kubernetes – a white paper

TIP: Not only we advise you to get familiar with these tools but also recommend you to have a look at two below resources to fully understand deployment part:

1. Stateful, Stateless, and Serverless – Running Apache Kafka on Kubernetes – online talk featuring Joe Beda, Heptio CTO and Kubernetes Co-creator, and Gwen Shapira, Principal Data Architect at Confluent.
2. Apache Kafka on Kubernetes – Could you? Should you? – this article gives the first reflection after the launch of Helm Charts.

You can already Sign up to Orchestrate Apache Kafka®on Kubernetes.

also posted here