Kafka 2.8 Introduces KRaft: Removing ZooKeeper with an Internal Quorum Controller

Apache Kafka’s upcoming 2.8 release adopts an internal Quorum controller, called KRaft, to replace ZooKeeper for metadata management.

This change allows users to run Kafka without any ZooKeeper instances, saving compute resources, improving performance, and enabling larger cluster sizes.

Historically, ZooKeeper acted as the coordination service for distributed systems like Kafka, handling broker registration and storing state changes, which added complexity to the overall architecture.

Since April of the previous year, work has progressed toward integrating the controller and ZooKeeper responsibilities into a single Quorum controller that runs inside the Kafka cluster; the feature is marketed as the Kafka Raft metadata mode (KRaft).

In KRaft mode, metadata previously split between the Kafka controller and ZooKeeper is merged and managed internally, though users can still deploy the Quorum controller on dedicated hardware for special scenarios.

The KRaft protocol uses an event‑driven mechanism to track cluster metadata, replacing many RPC‑based tasks with log‑based event propagation, which helps Kafka support more partitions.

Removing ZooKeeper makes Kafka lighter and more suitable for small‑scale workloads, edge devices, and lightweight hardware solutions.

However, the early‑access version still lacks several features such as ACLs, security, transactions, partition reassignment, and JBOD support; the developers plan to add these in later releases, and they advise against using KRaft in production environments at this stage.

References: https://www.confluent.io/blog/kafka-without-zookeeper-a-sneak-peek/ https://www.ithome.com.tw/news/143569