Understanding RocketMQ’s Network Communication Architecture and Protocol

1 Introduction

The previous chapters covered NameServer principles, message production, and consumption. RocketMQ’s architecture consists of four core components—NameServer, Broker (Master/Slave), Producer, and Consumer—whose interaction follows a defined workflow.

NameServer starts first as the central registry; brokers must register with it before launching.

After starting, a Broker registers to NameServer and maintains a long‑lived connection, sending a heartbeat every 30 seconds to confirm its liveliness and reporting its IP, port, and stored topics.

If NameServer detects a broker has been silent for more than 120 seconds, it removes the broker from the routing table.

Before sending a message, a Producer fetches the list of broker addresses from NameServer, selects one via load‑balancing, establishes a channel, and sends the message.

Before subscribing to a topic, a Consumer also obtains the broker list and topic‑queue mapping from NameServer, then connects directly to the appropriate broker to consume data.

Both Producer and Consumer refresh the broker list and topic‑queue information from NameServer every 30 seconds, ensuring up‑to‑date routing information for high throughput and stability.

2 Network Communication Process Analysis

2.1 Structure of the Communication Module (rocketmq‑remoting)

The rocketmq‑remoting module is dedicated to network communication. It is depended upon by rocketmq‑client , rocketmq‑broker , and rocketmq‑namesrv . The communication layer extends Netty and defines a custom protocol for transmitting messages between client and server.

2.2 Protocol Structure Design

RocketMQ defines its own message protocol via the RemotingCommand class. Important fields include:

code (int) : request operation code; determines business handling.

language (LanguageCode) : enumeration of the requestor’s programming language (JAVA, CPP, PYTHON, GO, etc.).

version (int) : version of the requestor’s program.

opaque (int) : unique request identifier.

flag (int) : distinguishes normal RPC (0) from one‑way RPC (1).

remark (String) : custom remark information.

extFields (HashMap<String, String>) : user‑defined extension fields for request or response.

2.3 Message Content Composition

The transmitted message consists of the following parts:

Message length : total length as a 4‑byte integer.

Serialization type + header length : 1 byte for serialization type, followed by 3 bytes for header length.

Header data : serialized header bytes.

Body data : binary payload of the message.

2.4 RocketMQ Message Communication Flow

RocketMQ supports three sending modes:

Sync – synchronous sending.

Async – asynchronous sending.

Oneway – fire‑and‑forget, no response required.

2.4.1 Communication Flow Explanation

The diagram below shows the initialization of NettyRemotingClient and NettyRemotingServer , message sending, and handler processing.

Broker and NameServer start by invoking NettyRemotingServer.start() , configuring BossGroup/WorkerGroup, channels, adding NettyServerHandler , and binding ports.

Producer and Consumer start the Netty client via NettyRemotingClient.start() , configuring client event loops, channels, and adding NettyClientHandler .

For synchronous messages, NettyRemoteClient.invokeSync() obtains or creates a channel from the cache.

The Producer writes and flushes data through the channel.

The server’s event loop reads the data and NettyServerHandler processes it.

processMessageReceived extracts the request code, selects the appropriate processor, and handles storage, lookup, or pull operations.

The response is matched with the original request via the opaque key and delivered to the client’s ResponseFuture (blocking for sync, callback for async).

The client’s NettyClientHandler reads the response and completes the request.

2.4.2 Reactor Multi‑Thread Design

RocketMQ’s communication relies on Netty’s Reactor model, enhanced with additional thread pools.

The model includes:

M : One Reactor main thread (eventLoopGroupBoss) that listens for TCP connections and registers channels.

S : Reactor selector thread pool (eventLoopGroupSelector) that reads incoming data (default 3 threads).

M1 : Worker thread pool (NettyServerCodecThread) handling SSL, idle checks, codec, serialization (default 8 threads).

M2 : Processor thread pool (RemotingExecutorThread) executing business logic based on RequestCode.

Thread mapping reference:

1 – NettyBoss (Reactor main thread)

N – NettyServerEPOLLSelector (Reactor selector pool)

M1 – NettyServerCodecThread (Worker pool)

M2 – RemotingExecutorThread (Processor pool)

Conclusion

The rocketmq‑remoting module is dedicated to network communication within RocketMQ.

It extends Netty and defines a custom protocol for structuring, encapsulating, and encoding/decoding message data.

Understanding the initialization of NettyRemotingServer / NettyRemotingClient and the role of their handlers is essential.

Synchronous vs. asynchronous messaging differs mainly in blocking response handling versus callback execution.

The communication flow utilizes one Reactor main thread, a Reactor selector pool, a worker pool, and a processor pool to achieve high performance and reliability.