Message Delivery Semantics in Distributed Systems: At-Most-Once, At-Least-Once, and Exactly-Once
The article explains how unreliable network communication in distributed systems leads to three message delivery semantics—at‑most‑once, at‑least‑once, and exactly‑once—detailing their mechanisms, ordering challenges, and common protocols such as AMQP and MQTT.
Network Requests
In distributed systems, network communication is inherently unreliable; a request may succeed, fail, or time out, and the caller cannot know the state of the remote node except through the response.
Success and Failure
When a network request returns a definitive result—either success or failure—the caller can handle it similarly to local IPC or method calls, provided the response was delivered without packet loss or server error.
Timeout
If a request does not receive a response within a configured timeout, it is considered failed; the caller cannot know whether the remote node processed the request, making retries risky for non‑idempotent operations.
Message Delivery Semantics
Three common semantics are used to describe how many times a message may be delivered:
At‑Most‑Once – the sender transmits the message a single time without retries; loss is possible.
At‑Least‑Once – the sender retries on timeout until an acknowledgement is received, guaranteeing delivery but allowing duplicates.
Exactly‑Once – theoretically impossible; implementations achieve it by combining at‑least‑once delivery with deduplication keys or idempotent processing.
Delivery Order
Unreliable networks can cause out‑of‑order delivery. Two common techniques to restore order are sequence numbers (similar to TCP SEQ) and state‑machine based processing.
Sequence Numbers
Each message carries an incremental sequence number; consumers can reorder or drop stale messages based on this value.
State Machine
Messages are treated as state‑transition events; the system defines allowed transitions so that even if messages arrive out of order, the final state remains consistent.
Messaging Protocols
Common message‑queue protocols provide standardized features for delivery semantics, routing, and persistence.
AMQP
Advanced Message Queuing Protocol defines queues, exchanges, transactions, and supports at‑most‑once and at‑least‑once semantics; implementations include RabbitMQ.
MQTT
Built on TCP/IP, MQTT is lightweight and offers three QoS levels that map directly to the three delivery semantics, handling retries and deduplication at the protocol layer.
Conclusion
Ensuring reliable message delivery in distributed systems requires handling network failures, timeouts, idempotency, and ordering; true exactly‑once delivery is unattainable, so systems combine at‑least‑once delivery with deduplication or idempotent processing to achieve practical reliability.
Architecture Digest
Focusing on Java backend development, covering application architecture from top-tier internet companies (high availability, high performance, high stability), big data, machine learning, Java architecture, and other popular fields.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.