Eight Common Use Cases of Message Queues in Backend Development
This article explores eight common scenarios for using message queues in backend development, covering asynchronous processing, service decoupling, traffic shaping, delayed tasks, log aggregation, distributed transactions, remote calls, and broadcast notifications, each illustrated with Java, RocketMQ, and Kafka code examples.
1. Asynchronous Processing: A Tool to Improve System Response Speed
Message queues enable asynchronous handling of tasks that would otherwise block the main flow, such as sending SMS or email after a user registers. By off‑loading these notifications to a queue, the registration API remains fast and resilient.
// User registration method
public void registerUser(String username, String email, String phoneNumber) {
// Save user information (simplified)
userService.add(buildUser(username, email, phoneNumber));
// Create notification message
String registrationMessage = "User " + username + " has registered successfully.";
// Send message to queue
rabbitTemplate.convertAndSend("registrationQueue", registrationMessage);
}Consumer code reads the message and performs the actual SMS/email sending.
@Service
public class NotificationService {
// Listen to the registration queue and send notifications
@RabbitListener(queues = "registrationQueue")
public void handleRegistrationNotification(String message) {
// Send SMS or email
System.out.println("Sending registration notification: " + message);
sendSms(message);
sendEmail(message);
}
}2. Decoupling: Breaking Strong Service Dependencies
In micro‑service architectures, direct synchronous calls create tight coupling. By publishing events to a queue, services such as payment and inventory can evolve independently.
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.common.message.Message;
public class PaymentService {
private DefaultMQProducer producer;
public PaymentService() throws Exception {
producer = new DefaultMQProducer("PaymentProducerGroup");
producer.setNamesrvAddr("localhost:9876"); // RocketMQ NameServer address
producer.start();
}
public void processPayment(String orderId, int quantity) throws Exception {
// Simulate payment call
boolean paymentSuccessful = callPayment(orderId, quantity);
if (paymentSuccessful) {
String messageBody = "OrderId: " + orderId + ", Quantity: " + quantity;
Message message = new Message("paymentTopic", "paymentTag", messageBody.getBytes());
producer.send(message);
}
}
}The inventory service consumes the payment event and reduces stock.
public class InventoryService {
private DefaultMQPushConsumer consumer;
public InventoryService() throws Exception {
consumer = new DefaultMQPushConsumer("InventoryConsumerGroup");
consumer.setNamesrvAddr("localhost:9876");
consumer.subscribe("paymentTopic", "paymentTag");
consumer.registerMessageListener((msgs, context) -> {
for (MessageExt msg : msgs) {
String messageBody = new String(msg.getBody());
reduceStock(messageBody);
}
return null; // indicate successful consumption
});
consumer.start();
System.out.println("InventoryService started...");
}
}3. Traffic Shaping: Handling High Concurrency Peaks
During spikes such as flash‑sale events, a queue can act as a buffer, allowing the system to process a limited number of requests per second and avoid overload.
4. Delayed Tasks: Precise Control of Execution Timing
For scenarios like order cancellation after a timeout, delayed queues let you schedule future processing without additional schedulers.
@Service
public class OrderService {
@Autowired
private RocketMQTemplate rocketMQTemplate;
public void createOrder(Order order) {
// Persist order (omitted)
long delay = order.getTimeout(); // milliseconds
// Send delayed message
rocketMQTemplate.syncSend("orderCancelTopic:delay" + delay,
MessageBuilder.withPayload(order).build(),
10000, // send timeout ms
(int) (delay / 1000) // RocketMQ delay level in seconds
);
}
}Consumer processes the delayed message to cancel the order.
@Component
@RocketMQMessageListener(topic = "orderCancelTopic", consumerGroup = "order-cancel-consumer-group")
public class OrderCancelListener implements RocketMQListener
{
@Override
public void onMessage(Order order) {
// Cancel order if still unpaid
System.out.println("Cancelling order: " + order.getOrderId());
// (actual cancellation logic omitted)
}
}5. Log Collection: Centralised Log Management
Applications can publish log entries to Kafka, where a log‑processing pipeline (ELK, Fluentd, etc.) aggregates and analyses them.
// Produce log to Kafka topic "app-logs"
KafkaProducer
producer = new KafkaProducer<>(config);
String logMessage = "{\"level\": \"INFO\", \"message\": \"Application started\", \"timestamp\": \"2024-12-29T20:30:59\"}";
producer.send(new ProducerRecord<>("app-logs", "log-key", logMessage));Consumer reads and handles the logs.
@Service
public class LogConsumer {
// Consume logs from Kafka
@KafkaListener(topics = "app-logs", groupId = "log-consumer-group")
public void consumeLog(String logMessage) {
System.out.println("Received log: " + logMessage);
}
}6. Distributed Transactions: Ensuring Data Consistency
By sending half‑transactions to the queue, the producer can commit or rollback based on local transaction outcome, guaranteeing eventual consistency across services.
Producer sends a half‑transaction message to MQ.
MQ stores the message in a pending state.
MQ acknowledges receipt without delivering.
Producer executes its local transaction.
On success, producer commits; on failure, it rolls back.
MQ updates the message status accordingly.
If committed, MQ pushes the message to consumers.
If MQ does not receive a final decision, it queries the producer to resolve the state.
7. Remote Calls: Building Efficient Communication Bridges
Using RocketMQ as a transport layer, a custom remote‑call framework can achieve high reliability and financial‑grade stability, supporting features such as multi‑center active‑active, gray releases, traffic weighting, deduplication, and back‑pressure.
8. Broadcast Notifications: Event‑Driven Messaging
MQ can broadcast events (e.g., order payment success) to multiple downstream systems like inventory, points, and finance services.
// Create order‑payment‑success event message
String orderEventData = "{\"orderId\": 12345, \"userId\": 67890, \"amount\": 100.0, \"event\": \"ORDER_PAYMENT_SUCCESS\"}";
Message msg = new Message("order_event_topic", "order_payment_success", orderEventData.getBytes());
producer.send(msg);Example listeners for each subsystem:
// Inventory system listener
consumer.registerMessageListener((MessageListenerConcurrently) (msgs, context) -> {
for (Message msg : msgs) {
String eventData = new String(msg.getBody());
System.out.println("Inventory system received: " + eventData);
// updateInventory(eventData);
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}); // Points system listener
consumer.registerMessageListener((MessageListenerConcurrently) (msgs, context) -> {
for (Message msg : msgs) {
String eventData = new String(msg.getBody());
System.out.println("Points system received: " + eventData);
// updateUserPoints(eventData);
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}); // Finance system listener
consumer.registerMessageListener((MessageListenerConcurrently) (msgs, context) -> {
for (Message msg : msgs) {
String eventData = new String(msg.getBody());
System.out.println("Finance system received: " + eventData);
// recordPaymentTransaction(eventData);
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
});The article concludes that mastering these eight MQ scenarios helps developers design robust, scalable, and maintainable backend systems.
Code Ape Tech Column
Former Ant Group P8 engineer, pure technologist, sharing full‑stack Java, job interview and career advice through a column. Site: java-family.cn
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.