Core Concepts and Architecture of RocketMQ

Before diving into RocketMQ, it is essential to clarify its core concepts to build a solid foundation for learning.

1. RocketMQ Deployment Architecture

The main components are:

Nameserver : A cluster that acts as the routing registry for topics, providing routing information to clients so they can send messages to the appropriate broker. Nameserver nodes do not communicate with each other, and routing data achieves eventual consistency.

Broker : The message storage server, which can be a Master or a Slave. In a typical 2‑master‑2‑slave setup, the master handles read/write operations while slaves act as backups and can serve read requests when the master is under pressure. Every broker (including slaves) sends a heartbeat to the Nameserver every 30 seconds, containing routing information for all topics it hosts.

Client : Includes producers (message senders) and consumers (message receivers). A client connects to one Nameserver at a time and only switches when the current connection fails. Clients query the Nameserver for topic routing every 30 seconds.

Tip: Nameserver stores topic routing information in memory; the persistent routing file resides in the broker at ${ROCKETMQ_HOME}/store/config/topics.json .

Since version 4.5.0, RocketMQ supports a multi‑replica mechanism where a replication group can be based on the Raft protocol, providing strong consistency for broker data—commonly used in the financial industry.

2. Message Subscription Model

RocketMQ uses a publish‑subscribe model.

Topic : A collection of messages of the same type, e.g., an order service publishes to order_topic , while login events go to user_login_topic .

ConsumerGroup : A group of consumers that jointly subscribe to one or more topics. Multiple consumer groups can subscribe to the same topic, and a single group can subscribe to multiple topics.

Example: An order system with a service order-service-app creates a consumer group order_consumer to subscribe to order_topic . The service runs on three servers, each JVM acting as a consumer within the group.

2.1 Consumption Modes

RocketMQ supports two consumption modes:

Broadcast Mode : Every consumer in the group processes every message in the topic, often used for cache refreshes.

Cluster Mode : Consumers share the workload; each message is processed by only one consumer in the group, enabling horizontal scaling.

2.2 Queue Load Algorithms and Rebalancing

In cluster mode, RocketMQ provides several queue allocation algorithms, the most common being:

AllocateMessageQueueAveragely (average allocation)

AllocateMessageQueueAveragelyByCircle (round‑robin allocation)

For a topic with 16 queues (q0‑q15) and three consumers (c0‑c2):

Average Allocation results in:

c0: q0 q1 q2 q3 q4 q5 c1: q6 q7 q8 q9 q10 c2: q11 q12 q13 q14 q15

Each consumer receives a contiguous block of queues, with the remainder distributed to the first consumers.

Round‑Robin Allocation results in:

c0: q0 q3 q6 q9 q12 q15 c1: q1 q4 q7 q10 q13 c2: q2 q5 q8 q11 q14

This method distributes queues cyclically.

Tip: If the number of queues is less than the number of consumers, some consumers will not receive messages. The queue count determines the maximum number of consumers, but increasing the queue count does not affect RocketMQ performance.

When a new consumer joins, RocketMQ automatically rebalances the queues without application intervention.

2.3 Consumption Progress

After a consumer processes a message, it records the offset so that, upon restart, consumption can resume from the last position. Offsets are stored per consumer group.

In cluster mode, offsets are stored on the broker. The file is {ROCKETMQ_HOME}/store/config/consumerOffset.json , with entries like topic@consumeGroup mapping each queue to its offset.

不能识别此Latex公式:
 { ROCKETMQ_HOME }/store/config/consumerOffset.json 是其具体的存储文件，其中内容截图如下：

可见消费进度的Key为：topic@consumeGroup，然后每一个队列一个偏移量。


广播模式的消费进度文件存储在用户的主目录，默认文件全路劲名：

For broadcast mode, offsets are stored in the user’s home directory at {USER_HOME}/.rocketmq_offsets .

2.4 Consumption Models

RocketMQ offers two consumption models:

Concurrent Consumption : Each consumer creates a thread pool to process messages from its assigned queues in parallel. Messages with larger offsets may be processed before those with smaller offsets.

Ordered Consumption : Guarantees message order within a queue (e.g., for MySQL binlog scenarios) by locking the queue while processing.

Tip: In concurrent mode, a failed message is retried up to 16 times with varying intervals. In ordered mode, a failed message is retried indefinitely until it succeeds; business‑level failures require alerting and manual intervention to avoid backlog.

3. Transaction Messages

Transaction messages ensure consistency between message sending and business data persistence, not to solve distributed transactions. The typical flow is illustrated below:

The pseudo‑code shows storing an order in a relational database and sending a message to MQ as a single atomic operation, achieved via RocketMQ’s transaction message feature.

Tip: This section only introduces terminology; detailed usage of transaction messages will be covered in subsequent articles.

4. Delayed Messages

The open‑source version of RocketMQ does not support arbitrary‑precision delayed messages. It provides a set of predefined delay levels:

1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h

5. Message Filtering

Consumers can filter messages within a topic based on conditions. RocketMQ supports tag‑based filtering and SQL92‑style property filtering.

6. Summary

This article covered fundamental RocketMQ terminology—Nameserver, Broker, Topic, Consumer Group, Queue Allocation Algorithms, Rebalancing, Consumption Progress Storage, Transaction Messages, Delayed Messages, and Message Filtering—laying a solid groundwork for future hands‑on tutorials.