Boost Throughput with RocketMQ Batch Messaging: How and Why

1 Background

In real-world internet services, bulk processing of messages is essential because large amounts of data are executed concurrently. When a business interaction triggers many asynchronous actions at different granularities, multiple MQ writes and connections are required, which is far less efficient than a single connection with batch writes.

2 Implementation of Batch Messages

RocketMQ batch messages improve throughput and processing efficiency, reduce downstream API call frequency, and enhance service stability.

2.1 Characteristics of Batch Messages

All messages share the same topic.

All messages share the same waitStoreMsgOK property.

Batch messages do not support delayed delivery.

Batch size must not exceed 4 MB (after version 4.4 the limit is 1 MB).

2.2 Use Cases

Increasing throughput by packing multiple messages into one batch, reducing network overhead and processing time.

Lowering downstream API call frequency by sending fewer, larger batches.

2.3 Sending Example

RocketMQ provides a batch‑send API via DefaultMQProducer.send() . The following Java example creates a producer, builds a list of three messages, and sends them as a batch.

<code>DefaultMQProducer producer = new DefaultMQProducer("BatchProducerGroupName_1");
String topic = "BatchSendTest_1";
producer.start();
List&lt;Message&gt; msgs = new ArrayList&lt;&gt;();
msgs.add(new Message(topic, "Tag1", "OrderID-063105013", "Hello world".getBytes()));
msgs.add(new Message(topic, "Tag1", "OrderID-063105014", "Brand".getBytes()));
msgs.add(new Message(topic, "Tag1", "OrderID-063105015", "handsome boy ".getBytes()));
try {
    producer.send(msgs);
} catch (Exception e) {
    e.printStackTrace();
} finally {
    producer.shutdown();
}</code>

If the total size may exceed 1 MB, the messages should be split. The following ListSplitter implementation demonstrates how to partition a large list into sub‑lists that respect the size limit.

<code>public class ListSplitter implements Iterator<List<Message>> {
    private final int SIZE_LIMIT = 1024 * 1024;
    private final List<Message> messages;
    private int currIndex;
    // constructor and iterator methods omitted for brevity
}</code>

Using the splitter, each sub‑list can be sent safely:

<code>ListSplitter splitter = new ListSplitter(messages);
while (splitter.hasNext()) {
    try {
        producer.send(splitter.next());
    } catch (Exception e) {
        e.printStackTrace();
    }
}</code>

Overall, RocketMQ batch messaging can significantly improve throughput, reduce downstream API calls, and serve as an effective optimization for message transmission.

3 Summary

Messages with the same type and characteristics can be aggregated into batches, reducing connection overhead and boosting performance.

Batch messages must share the same topic and waitStoreMsgOK property and cannot be delayed messages.