Choosing the Right Message Queue: Kafka vs RabbitMQ vs RocketMQ vs ActiveMQ Explained

RabbitMQ, Kafka, RocketMQ and ActiveMQ: their principles, how to choose, and detailed comparisons.

Message‑queue middleware is a frequent interview topic. This article provides a comprehensive theoretical overview, gathered from various sources over a month, to help you answer why you would choose one MQ over another.

Message Queue

Message Queue Patterns

Two main patterns exist: point‑to‑point and publish/subscribe.

Point‑to‑Point

Each message is consumed by a single consumer. Multiple producers can send to the same queue, but once a consumer processes a message it is locked or removed, preventing other consumers from handling it. If processing fails, the message is usually returned to the queue for retry.

Publish/Subscribe

A single message can be delivered to multiple subscribers. Subscriptions are either:

Ephemeral – exists only while the consumer is running; lost when the consumer stops.

Durable – persists until explicitly deleted, allowing the broker to retain the subscription after consumer shutdown.

Evaluation Criteria

When selecting a message queue, consider the following metrics:

Message order – can the system guarantee processing order?

Message routing – ability to filter messages based on routing rules.

Reliability – risk of message loss.

Timing – TTL, delayed or scheduled messages.

Retention – whether messages remain after successful consumption.

Fault tolerance – mechanisms to ensure successful processing.

Scalability – ability to scale up/down under load.

Throughput – maximum concurrent processing capacity.

Message Queue Comparison

The following chart (originally sourced) compares mainstream MQs.

Brief introductions of the four commonly used queues:

Kafka : Distributed commit‑log system originally from LinkedIn, excels in high‑throughput data ingestion for big‑data scenarios.

RabbitMQ : Erlang‑based, AMQP‑compliant broker focused on reliability and routing, suitable for enterprise systems requiring strong consistency.

RocketMQ : Alibaba’s Java‑based broker, high throughput and availability, optimized for large‑scale distributed applications.

ActiveMQ : Apache’s widely used broker, now less maintained for large‑scale throughput.

Pros & Cons

Kafka

Advantages:

High throughput, low latency (tens of thousands of messages per second).

Excellent scalability via partitions.

Persistent storage with replication via Zookeeper.

Fault‑tolerant distributed architecture.

Message ordering within a partition.

Rich ecosystem (e.g., Kafka‑Manager).

Simple core functionality, widely adopted in real‑time analytics.

Disadvantages:

Performance degrades with many partitions per broker.

Short‑polling leads to latency dependent on poll interval.

No built‑in retry for failed consumption.

Partition leader failure can cause ordering issues.

Community updates slower than some alternatives.

RabbitMQ

Advantages:

Supports many protocols and languages.

High availability clustering.

Pluggable authentication, TLS, LDAP.

Extensible via plugins.

User‑friendly management UI.

Disadvantages:

Erlang codebase can be hard to understand for custom extensions.

Throughput lower than Kafka due to heavier implementation.

Steeper learning curve for APIs and protocols.

RocketMQ

Advantages:

Supports both Pub/Sub and P2P models.

FIFO guarantee within a queue.

Pull and push consumption modes.

Million‑message queue capacity.

Multiple protocol support (JMS, MQTT, etc.).

Horizontal scaling with strict at‑least‑once semantics.

Docker images for isolated testing.

Rich dashboard for metrics and monitoring.

Disadvantages:

Client language support limited to Java and C++.

Community activity moderate.

Lacks native JMS implementation, requiring code changes for migration.

ActiveMQ

Advantages:

Cross‑language client support.

Full JMS 1.1 / J2EE 1.4 compliance.

Advanced features like virtual destinations and message groups.

Spring integration.

High‑performance clustering design.

Supports multiple transport protocols (TCP, SSL, NIO, UDP, etc.).

Fast persistence via JDBC logging.

Disadvantage:

Community maintenance for 5.x declining; not ideal for massive throughput.

Kafka Deep Dive

Kafka is a distributed, partitioned, replicated log platform built on Zookeeper, using a publish/subscribe model.

Core Concepts

Message – a record stored in a topic.

Batch – a group of messages sent together for efficiency.

Topic – logical category of messages, similar to a database table.

Partition – a sub‑segment of a topic that enables scalability; ordering is guaranteed only within a partition.

Producer – client that publishes messages to a topic.

Consumer – client that subscribes to a topic.

Consumer Group – a set of consumers that share the load of a topic.

Producer Workflow

1. Create a

ProducerRecord

(topic, optional partition, key/value). 2. Serialize key/value to bytes. 3. Determine target partition (explicit or key‑hash). 4. Batch records per partition. 5. Send batch via a dedicated thread. 6. Broker returns

RecordMetadata

(topic, partition, offset, timestamp) or an error.

Summary: create object → serialize → route to partition → batch → send → receive metadata.

Partition Strategies

Round‑robin (default) – evenly distributes messages across partitions.

Random – less efficient than round‑robin.

Key‑hash – guarantees that messages with the same key land in the same partition, preserving order for that key.

Consumer Mechanics

Consumers belong to a group; each group receives the full stream, while partitions are divided among the group's members. Adding consumers increases parallelism; however, the number of consumers should not exceed the number of partitions.

To consume the full message stream, assign a dedicated consumer group; increase group size to boost throughput.

Rebalancing

When consumers join or leave a group, partitions are reassigned (rebalance). During rebalance, consumption pauses, which can cause temporary unavailability.

Feature Analysis

Message routing – not supported (Kafka delivers all messages in a partition).

Ordered consumption – supported within a partition; failures break ordering.

Message timing – no built‑in delay or TTL.

Fault tolerance – high at cluster level, but no automatic retry for failed messages.

Scalability – excellent via partitions and consumer groups.

Persistence – strong; messages remain on disk.

Replay – possible thanks to persistence.

Throughput – very high due to partitioned design.

RocketMQ Overview

RocketMQ is a pure‑Java, distributed queue model open‑source middleware originally derived from MetaQ, now an Apache top‑level project.

Basic Concepts

NameServer – registration service similar to Zookeeper.

Producer – generates messages.

Consumer – processes messages.

Broker – stores and forwards messages.

Message – payload with optional tag and key.

Topic – primary classification of messages.

Tag – secondary classification.

Group – collection of producers or consumers.

Queue – ordered sub‑segment within a topic (equivalent to Kafka partition).

Message Model

Publishers send messages to a topic; the topic contains multiple queues. Consumers in a group pull from these queues, and only one consumer per queue processes messages. Off‑line consumers are taken over by others in the group.

System Architecture

Broker – stores, delivers, and provides high availability.

NameServer – decentralized registry for broker routing information.

Producer – publishes messages, can be clustered.

Consumer – pulls or pushes messages, supports cluster or broadcast modes.

Advanced Features & Common Issues

Ordered Consumption

Order is guaranteed only within a single queue. To maintain order for related messages (e.g., an order’s lifecycle), hash the business key to the same queue.

Duplicate Consumption

Idempotency must be handled at the application level (e.g., using Redis or database unique keys) to avoid side effects from retries.

Distributed Transactions

RocketMQ provides transactional messages (half‑message, commit/rollback, and transaction check) to achieve atomicity across services.

Message Accumulation

Messages can accumulate in memory buffers or be persisted to disk; the impact on throughput depends on the storage medium.

Scheduled Messages

RocketMQ supports delayed delivery at predefined levels (e.g., 5 s, 10 s, 1 min) but not arbitrary timestamps.

Persistence Strategies

Both synchronous and asynchronous flushes are available. Synchronous flush ensures durability at the cost of latency; asynchronous flush improves performance but may lose data on broker crash.

Replication

Synchronous replication – writes must be confirmed on both master and slave before success.

Asynchronous replication – master returns success immediately; slaves catch up later, affecting availability but not durability.

Fault Tolerance

RocketMQ uses an ACK mechanism; if a consumer does not acknowledge success, the broker retries the message (default 10 s delay) up to a configurable limit, after which it moves the message to a dead‑letter queue.

Feature Analysis

Message routing – not supported.

Ordered consumption – partially supported (requires same‑queue hashing).

Message timing – supports delayed messages at fixed levels.

Fault tolerance – ACK‑based retry and DLQ.

Scalability – similar to Kafka; can add brokers, queues, and consumer instances.

Persistence – disk‑based, enabling replay.

High throughput – leverages partitioned design for massive parallelism.

RabbitMQ Overview

RabbitMQ, released in 2007, is an Erlang‑based open‑source broker implementing the AMQP protocol, emphasizing reliability, flexible routing, and broad language support.

For hands‑on practice, see the linked "RabbitMQ入门" article.