0 views collected around this technical thread.
This article explains the three message delivery semantics, the common causes of duplicate messages in queue systems, and presents concrete producer‑side, broker‑side, and consumer‑side deduplication techniques for Kafka and Pulsar, including code samples and best‑practice recommendations.
This article explains common causes of duplicate message consumption in high‑traffic systems and presents a three‑layer defense—producer idempotence, broker de‑duplication, and consumer idempotent design—plus monitoring and reconciliation strategies to achieve reliable exactly‑once processing.
Message queues can deliver duplicates under at‑least‑once semantics, so to protect idempotent business logic such as orders and payments you should combine producer‑side idempotence (e.g., Kafka’s enable.idempotence), broker‑side deduplication (e.g., Pulsar), and a consumer‑side guard using unique IDs stored in a database or Redis.
Message queues can cause duplicate messages that affect idempotent business processes, so this article explains the three delivery semantics (At Least Once, Exactly Once, At Most Once), the causes of duplication, and practical deduplication techniques for producers, brokers (Kafka, Pulsar), and consumers using code examples.
This article shares hard‑won lessons from production incidents and provides practical guidelines—graceful shutdown, proper startup timing, suitable business scenarios, reliability patterns, and idempotent handling—to use Spring Event safely and effectively in Java backend systems.
This article explains the concept of idempotence, why it matters in distributed systems, how to handle timeout scenarios, design principles using globally unique IDs, and eight practical implementation patterns—including database tricks, token schemes, locking, and HTTP method considerations—to build robust, repeat‑safe APIs.
The article explains the concept of idempotence in computing, illustrates typical duplicate‑request scenarios, and presents three practical backend solutions—unique database indexes, optimistic locking with version control, and a Redis‑based token mechanism—along with their advantages, trade‑offs, and implementation details.
Idempotence ensures that repeating the same operation yields the same result without side effects, a critical property in distributed and transactional systems; the article defines the concept, explains why it matters, lists common causes of non‑idempotent behavior, and presents several practical backend solutions such as unique indexes, anti‑duplicate tables, optimistic and pessimistic locks, tokens, distributed locks, and state machines.
This article explains why message duplication occurs in Kafka, describes the three delivery semantics, and provides practical solutions—including idempotent producers, transactions, and consumer-side idempotence—along with configuration tips and code examples to achieve exactly‑once delivery.
This article explains Kafka's internal topics, preferred replicas, partition assignment processes, log directory layout, index files, offset and timestamp lookup, log retention and compaction policies, storage architecture, delayed operations, controller role, legacy consumer design flaws, rebalance workflow, and producer idempotence, providing a comprehensive overview of Kafka's backend architecture.
This article explains why message middleware guarantees at‑least‑once delivery, describes three common duplication scenarios in RocketMQ, and presents both transactional and non‑transactional deduplication solutions—including SQL examples and a Redis‑based idempotence library—to achieve exactly‑once processing.
This article explains common scenarios that cause message loss or non‑idempotent processing in RabbitMQ and Kafka, and presents practical solutions such as persistent delivery, confirm mechanisms, delayed delivery, and unique‑ID plus fingerprint strategies to achieve reliable and idempotent message transmission.
This article explains Kafka’s three delivery semantics—At most once, At least once, and Exactly once—from both producer and consumer perspectives, details the required configuration settings, and discusses how Kafka ensures idempotence, transaction support, and prevents data loss, duplication, and ordering issues.
This article explains Kafka's exactly‑once semantics and transaction support, detailing the new producer API methods, related exceptions, configuration parameters, and a sample application illustrating how to initialize, begin, process, and commit or abort transactions while ensuring idempotent and atomic message handling.
This article explains how to build a high‑availability service layer for large‑scale web systems by vertically splitting business domains, adopting stateless design, configuring timeouts, using asynchronous messaging, ensuring idempotent operations, and applying service degradation techniques to maintain stability during traffic spikes.
This article reviews typical Kafka interview topics, explaining the implementation of delay queues with hierarchical time wheels, how idempotence is achieved via producer IDs, the meanings of ISR, OSR, AR, HW, LEO, LSO, strategies for guaranteeing message order, and practical solutions for handling duplicate consumption.
This article explains common scenarios where Kafka messages can be lost on the producer, consumer, or broker side, and provides practical configurations—including callbacks, acks, retries, manual offset commits, idempotent and transactional producers—to ensure reliable delivery and exactly‑once processing.
This article reviews common Kafka interview questions, covering delay queues, idempotence, replica states, offsets, message ordering, and handling duplicate consumption, and includes example code for enabling idempotent producers along with explanations of time‑wheel mechanisms and practical solutions to consumer rebalance issues.
This article explains Kafka's Exactly-Once Semantics (EOS), the role of idempotence, and how transactional support works, covering EOS semantics, producer id and sequence numbers, configuration properties, and providing Java code examples for initializing, beginning, committing, and aborting transactions.