Understanding Message Middleware: Queue and Publish‑Subscribe Styles

Message middleware has become widely used in many companies for decoupling services, smoothing traffic spikes, integrating with big data, and enabling stream processing. Fundamentally, it involves two steps: producing messages and consuming them, acting as a temporary (and sometimes persistent) storage that provides high throughput, reliability, and distributed scalability.

The middleware can move messages from producers to consumers in two main styles: Message Queuing and Publish‑Subscribe.

Message Queuing works like a work‑thread pool with a task queue: tasks are enqueued and idle worker threads (consumers) pull tasks one by one. Each task is processed by only one consumer, and the order of processing may differ from the order of insertion. To avoid duplicate consumption, messages are typically deleted or marked after processing. Scaling the number of consumers can increase throughput when the queue backs up.

Publish‑Subscribe (pub‑sub) pushes messages to a topic; every subscriber receives the full set of messages in the same order they were published. This model guarantees ordered delivery, which is useful for scenarios like synchronizing database binlogs. It is often favored by stateless applications that prioritize concurrency and throughput over strict ordering.

Common message middleware includes:

Apache ActiveMQ, Amazon SQS, IBM WebSphere MQ, RabbitMQ, and RocketMQ – primarily support the queue style.

Apache Kafka – supports both styles; within a consumer group it behaves like a queue, while different consumer groups act as independent subscribers, providing a pub‑sub experience.

When selecting a middleware, consider the specific business requirements and trade‑offs of each style.