Understanding Apache RocketMQ Architecture: Components, Routing, and Message Flow

Apache RocketMQ is an open‑source, high‑performance distributed messaging middleware developed by Alibaba. It consists of three main parts: Producer (generates messages), Broker (stores and forwards messages), and Consumer (receives messages). A lightweight Namesrv cluster provides routing information and broker registration.

Overall Architecture – Namesrv acts as a naming service, maintaining metadata such as topicQueueTable , brokerAddrTable , clusterAddrTable , brokerLiveTable , and filterServerTable . Producers and consumers query Namesrv for topic routes, while brokers periodically send heartbeats to update their status.

Namesrv Functions – It supplies topic routing to producers/consumers, manages broker nodes, registers and deregisters routes based on heartbeat timestamps, and removes stale broker information after a timeout.

Broker Role – Brokers store messages, maintain metadata (consumer groups, offsets, topic‑queue mappings), and can be organized into master/slave groups. Each broker connects to all Namesrv nodes and reports its health and topic configuration.

Producer Role – Producers are stateless, can be clustered, and obtain topic routes from a randomly chosen Namesrv. They establish long connections with the master broker of the target topic and support synchronous, asynchronous, ordered, and one‑way sending modes.

Consumer Role – Consumers also connect to a random Namesrv to fetch routing information, then pull messages from the appropriate master/slave brokers. Two consumption models exist: pull (application initiates fetch) and push (broker pushes via long‑polling). In a consumer group, each queue is assigned to only one consumer instance to avoid duplicate consumption.

Consumer Types – Pull Consumer (application‑driven) and Push Consumer (broker‑driven, essentially long‑polling).

Key Concepts – Topic (a collection of messages, divided into queues), Tag (sub‑category within a topic), Producer Group , Consumer Group , Message (with unique ID and optional key), and ordering guarantees (normal vs. strict).

Advanced Features – RocketMQ does not natively support message priority but can emulate it by using separate high‑priority queues. It guarantees strict ordering when required and provides high availability through broker master/slave replication and persistent commit logs.

Scalability & Parallelism – Parallelism is increased by adding brokers and queues. Production parallelism depends on broker count; consumption parallelism depends on queue count and consumer thread configuration. Broadcast mode delivers all messages to every consumer, while clustering mode distributes queues among consumers.

Queue Allocation Strategies – Producers select a queue via MessageQueueSelector (e.g., by hash, random, machine‑room). Consumers use AllocateMessageQueueStrategy to assign queues to consumer instances. Built‑in strategies include average allocation, ring‑based allocation, machine‑room proximity, consistent hash, and configurable allocation.

Comparison with Kafka – Both use a naming service (Namesrv vs. ZooKeeper) and support partitioned topics. Kafka’s partitions are managed by ZooKeeper, while RocketMQ introduces the broker‑group concept. Both allow dynamic topic creation and scaling.

For further reading, see the official RocketMQ documentation and community articles linked at the end of the original post.