Redpanda vs Apache Kafka with KRaft: Why Redpanda Is Up to 10× Faster

Test Method

We used the Linux Foundation’s OpenMessaging Benchmark to fairly compare Redpanda and Kafka, leveraging Terraform and Ansible for reproducible cluster setups. Tests ran with TLS encryption and SASL authentication to reflect real‑world deployments, and Kafka was configured to sync all messages via fsync for data safety.

Note: KRaft does not support SASL, so KRaft tests were run without SASL, giving Kafka a slight performance edge in those runs.

KRaft Overview

Kafka 3.4.0 introduces KRaft, a Raft‑based metadata management system that replaces Apache ZooKeeper. While KRaft handles cluster metadata, data replication still relies on the traditional ISR protocol. Limitations of KRaft include no ZooKeeper‑to‑KRaft upgrade path, lack of SASL/SCRAM support, and required changes to TLS‑incompatible CLI tools.

Redpanda uses Raft for both metadata and data replication, providing stronger safety guarantees and avoiding the leader election issues that can affect Kafka.

The diagram illustrates how Kafka’s topology changes when moving from ZooKeeper to KRaft, adding extra controller nodes, whereas Redpanda’s Raft is built‑in and requires no external arbitration service.

Test: Redpanda 23.1 and Kafka 3.4.0 with KRaft

3.1 Enabling fsync

Benchmarks were run on three AWS instance types (i3en.6xlarge, i4i.8xlarge, is4gen.4xlarge). Redpanda achieved the 1 GBps workload on a three‑node cluster for all instance types. Kafka required additional brokers (and ZooKeeper or KRaft controllers) to match Redpanda’s throughput, and on Graviton instances Kafka could not sustain the workload when TLS was enabled.

Broker counts needed for each workload:

i3en.6xlarge – Redpanda: 3 brokers, Kafka: 6 brokers (+3 KRaft/ZooKeeper) i4i.8xlarge – Redpanda: 3 brokers, Kafka: 4 brokers (+3 KRaft) is4gen.4xlarge – Redpanda: 3 brokers, Kafka: unable to complete

Redpanda’s latency results (median E2E and P99.9) were consistently lower than Kafka’s. For example, on i3en.6xlarge Redpanda’s P99.9 latency was 24.376 ms versus Kafka’s 97.951 ms (≈4× faster). On i4i.8xlarge the gap widened to ≈20×.

3.2 Disabling fsync

When fsync is disabled, Kafka’s tail‑latency degrades further, and data loss risk increases. Redpanda, which only acknowledges writes after a majority of replicas have persisted to disk, maintains its safety guarantees while still outperforming Kafka in latency.

Conclusion: Redpanda Is Faster Regardless of Comparison

Key takeaways from the latest benchmarks:

Redpanda handles higher GBps‑level streaming workloads on the same hardware.

Even with 2–3× additional hardware, Apache Kafka’s P99.9 latency remains 4–20× slower than Redpanda’s.

When Kafka is run without fsync (trading safety for speed), its P99.9 latency is still 3–8× slower than Redpanda’s.

Thanks to its per‑core thread architecture and efficient hardware utilization, Redpanda can reduce latency by up to tenfold compared to Kafka. Users are encouraged to run these benchmarks themselves; detailed instructions are available in the benchmark guide.