Key Points for Revisiting Message Queue Middleware: Usage, Drawbacks, Selection, High Availability, Idempotency, Reliability, and Ordering

Introduction

The author writes this article to help two friends, A and B, who work in traditional software outsourcing and a state‑owned enterprise respectively, to review the key points of message‑queue middleware.

Why write this article?

Friend A works on daily product requirements, business‑logic tweaks, occasional SQL reports, and occasional bug‑fix deployments, experiencing zero technical growth. Friend B, although exposed to middleware, only knows how to publish/subscribe APIs without understanding why or how to ensure high availability.

Both are eager to improve, so the author provides a concise review of message‑queue fundamentals.

Review Points

The article is organized around the following seven questions:

Why use a message queue?

What are the drawbacks of using a message queue?

How to select a message queue?

How to guarantee high availability?

How to avoid duplicate consumption?

How to ensure reliable transmission?

How to preserve message ordering?

The article is a review guide, not a full‑fledged tutorial.

Main Content

1. Why use a message queue?

Analysis: Without understanding the purpose, one may be unable to answer interview questions and end up speaking nonsense.

Answer: The three main scenarios are decoupling, asynchrony, and peak‑shaving .

(1) Decoupling

Traditional mode:

Drawbacks: Tight coupling between systems; adding a new system requires code changes.

Middleware mode:

Advantages: Systems publish to the queue and subscribe independently, avoiding code modifications.

(2) Asynchrony

Traditional mode:

Drawbacks: Synchronous execution of non‑essential logic consumes time.

Middleware mode:

Advantages: Non‑essential tasks run asynchronously, improving response speed.

(3) Peak‑shaving

Traditional mode:

Drawbacks: High concurrency drives all requests directly to the database, causing connection failures.

Middleware mode:

Advantages: Consumers pull messages at a rate the database can handle, allowing temporary peaks to be buffered.

2. What are the drawbacks of using a message queue?

Analysis: Introducing MQ without understanding its risks can create project pitfalls.

Reduced system availability: If the MQ fails, dependent services also fail.

Increased system complexity: Additional concerns such as consistency, duplicate consumption, and reliable transmission must be handled.

Nevertheless, MQ is often necessary.

3. How to select a message queue?

The author is familiar with ActiveMQ, RabbitMQ, RocketMQ, and Kafka, and compares them.

ActiveMQ community activity:

Apache ActiveMQ 5.15.3 Release
Christopher L. Shannon posted on Feb 12, 2018
Apache ActiveMQ 5.15.2 Released
...

RabbitMQ release frequency:

RabbitMQ 3.7.3 release 30 January 2018
RabbitMQ 3.6.15 release 17 January 2018
RabbitMQ 3.7.2 release 23 December 2017
...

RabbitMQ updates more frequently than ActiveMQ. RocketMQ and Kafka are even more active.

Feature

ActiveMQ

RabbitMQ

RocketMQ

Kafka

Development language

Java

Erlang

Java

Scala

Single‑node throughput

10k+

10k+

100k+

100k+

Latency

ms

µs

ms

within ms

Availability

High (master‑slave)

High (master‑slave)

Very high (distributed)

Very high (distributed)

Feature set

Mature, many docs, good protocol support

Strong concurrency, excellent performance, rich UI

Complete MQ features, good extensibility

Core MQ features, oriented to big‑data scenarios

Recommendations:

For small‑to‑medium software companies, choose RabbitMQ for its Erlang‑based high concurrency and convenient UI.

For large enterprises, consider RocketMQ or Kafka depending on data volume and specific use cases (e.g., log collection favors Kafka).

4. How to guarantee high availability?

High‑availability requires understanding cluster modes.

RocketMQ example: Multi‑master, multi‑master‑slave asynchronous replication, and synchronous double‑write modes. Architecture diagram:

Producer connects to a NameServer node to obtain routing info, then to the broker master. Consumer connects to both master and slave for redundancy.

Kafka example: Uses Zookeeper for metadata and leader election. Typical diagram:

RabbitMQ also offers normal and mirrored cluster modes.

5. How to avoid duplicate consumption?

Duplicate consumption occurs when acknowledgments are lost. Solutions vary by MQ:

Use a unique primary key in the database to make inserts idempotent.

Leverage idempotent operations such as Redis SET.

Record consumption in a third‑party store (e.g., Redis) and check before processing.

6. How to ensure reliable transmission?

Reliability must address producer loss, broker loss, and consumer loss.

RabbitMQ

Producer loss: Use transaction or confirm mode. Example confirm‑listener code:

channel.addConfirmListener(new ConfirmListener() {
    @Override
    public void handleNack(long deliveryTag, boolean multiple) throws IOException {
        System.out.println("nack: deliveryTag = " + deliveryTag + " multiple: " + multiple);
    }
    @Override
    public void handleAck(long deliveryTag, boolean multiple) throws IOException {
        System.out.println("ack: deliveryTag = " + deliveryTag + " multiple: " + multiple);
    }
});

Broker loss: Enable durable queues (durable=true) and persistent messages (deliveryMode=2).

Consumer loss: Avoid auto‑ack; use manual acknowledgments.

Kafka

Producer configuration: acks=all and retries=MAX to prevent loss.

Broker configuration: set replication.factor > 1 and min.insync.replicas > 1 to ensure data is replicated before acknowledgment.

Consumer configuration: disable auto‑commit and commit offsets manually after processing.

7. How to guarantee message ordering?

Place ordered messages into the same queue/partition and let a single consumer process them. If multiple consumers are needed, design the business logic to tolerate out‑of‑order handling or implement retry mechanisms.

Conclusion

After reviewing these seven questions, readers should be able to cover most interview topics related to message queues. The author encourages deep preparation, solid fundamentals, and a thoughtful, analytical approach to software development.

Original source: cnblogs.com/rjzheng/p/8994962.html