Message Queue (MQ) Usage Scenarios, Patterns, and Pitfalls

Decoupling

Message queues are most directly used to decouple two systems. For example, in an order‑payment deduction scenario, the order service publishes a message to MQ and immediately returns to the user, while the settlement service consumes the message to deduct the user's account balance. This separation allows each system to focus on its own responsibilities and improves throughput, though it introduces retry and idempotency concerns.

Peak‑Shaving (削峰填谷)

During high‑traffic promotions, an order service may generate orders at a rate that overwhelms the settlement service. By publishing orders to MQ and letting the consumer pull messages at a controlled pace, traffic is smoothed, preventing the settlement service from being overloaded and achieving flow control.

The pull model requires the consumer to invoke a pull method manually, giving the application full control over consumption speed. Below is a code example:

messageConsumer.start();
for (;;) {
    // 手动拉取消息
    messageConsumer.pull(topic, messageListener);
}

In this model, the consumer specifies the topic and a callback listener; the listener’s onMessage() method is invoked for each retrieved message. Unlike the listener mode, the pull mode does not require a daemon thread and does not need explicit message acknowledgments, making it suitable for write‑heavy scenarios where the result only needs to be persisted.

Eventual Consistency (最终一致性)

Internet‑scale services often aim for eventual consistency rather than strong consistency. Continuing the order‑settlement example, the order creation and the MQ notification can be wrapped in a local transaction so that both succeed or both fail. If MQ delivery fails, a scheduled compensation task can retry, ensuring the order result is eventually persisted. The settlement consumer relies on MQ’s retry mechanism and idempotent processing to achieve final consistency.

Broadcast Consumption (广播消费)

MQ supports two messaging patterns: point‑to‑point and publish/subscribe. Within publish/subscribe, consumption can be clustered (only one consumer in the group processes each message) or broadcast (every consumer in the group receives each message). Broadcast consumption is useful for scenarios like push notification services, where each server in a cluster must be aware of a message to determine whether a connected client exists locally.

Simulating Broadcast with Cluster Consumption

Because broadcast mode cannot guarantee ordered messages and may cause higher duplicate rates, a common workaround is to simulate broadcast using clustered consumption. By assigning a distinct consumer group identifier (APPID) to each server while keeping the subscription identical, each server receives a copy of the message, achieving broadcast‑like behavior without the drawbacks of true broadcast mode.

Retry Pitfalls (重试之坑)

MQ’s retry mechanism ensures that messages are eventually processed, but it places the burden of idempotency on the business logic. For example, in a payment‑deduction flow, if the order service retries a request after a network failure, the settlement service might process the same transaction twice unless it validates a unique transaction identifier and discards duplicates.

Conclusion

The article covered common MQ usage scenarios—decoupling, peak‑shaving, eventual consistency, broadcast consumption, and retry handling—along with practical considerations such as idempotent design, consumption monitoring, and the trade‑offs between broadcast and clustered consumption. Understanding these patterns helps developers leverage MQ effectively while avoiding consistency and duplication issues.