Understanding Mutual Exclusion and Idempotency in Distributed Systems: Distributed Locks, ReentrantLock, Zookeeper, Redis, and GTIS

With the rapid growth of internet services, high concurrency, massive data, and complex business logic have become common, making low‑cost high‑availability and scalable system design increasingly important.

Mutual‑exclusion problem

Two typical scenarios illustrate the issue: (1) concurrent updates to a shared counter and (2) multiple processes pulling tasks from the same pool, both of which can lead to inconsistent results when no coordination is applied.

In traditional centralized systems, database transactions (ACID) solve resource contention, but in distributed environments additional mechanisms are required.

Multi‑thread and multi‑process solutions

In Java, ReentrantLock uses CAS and a CLH queue to provide fair or non‑fair locking, while the synchronized keyword relies on monitorenter and monitorexit bytecode instructions. The JVM optimizes these locks to avoid costly kernel‑mode mutexes.

For multi‑process coordination, operating‑system semaphores (named or unnamed) provide P/V operations to protect critical sections.

Distributed lock basics

A distributed lock must satisfy three core conditions: a globally accessible storage space, a unique identifier for each lock, and at least two distinct states (locked/unlocked). Common storage back‑ends include databases, Redis, Tair, and Zookeeper.

Typical implementations

ZooKeeper

ZooKeeper offers ephemeral sequential nodes. Clients create a node under a lock path, retrieve the list of children, and acquire the lock if their node has the smallest sequence number; otherwise they watch the predecessor node.

Lock release simply deletes the client’s node.

Redis

Redis implements a lock using SETNX (or GETSET ) to ensure atomic acquisition, often combined with an expiration time to avoid deadlocks caused by crashes.

SETNX lock.key 1
EXPIRE lock.key 30

Redisson provides a more complete solution built on top of Redis.

Tair

Tair’s expireLock method stores a lock status together with a timestamp, allowing atomic checks without relying on synchronized clocks.

All these approaches share the same fundamental steps: store a lock identifier, attempt atomic acquisition, handle failures, and clean up on release.

Cerberus distributed lock

Cerberus abstracts multiple engines (Zookeeper, Tair, future Redis) behind a unified API, supports fair and non‑fair locking, and provides a one‑click downgrade mechanism to switch engines when a primary engine fails.

Key methods mirror java.util.concurrent.locks.Lock (e.g., lock() , tryLock() , unlock() ).

Idempotency problem

In distributed systems, duplicate requests (due to UI retries, network glitches, or message re‑delivery) can cause inconsistent state. Guarantees of exactly‑once delivery are impossible, so services must enforce idempotency.

GTIS (Global Transaction Idempotency Service)

GTIS generates a global ID for each business operation by hashing a unique operation descriptor (e.g., order ID) with MD5. The ID is stored in an external store (Tair) using SETNX to guarantee atomicity.

final boolean lock = tair.setNx(globalId, timestamp + content);
if (!lock) { /* duplicate, reject */ }

If the operation succeeds, GTIS extends the key’s TTL; if it fails or times out, the key is removed, allowing retries.

GTIS also handles failure scenarios such as host crashes, network partitions, and clock skew by using timestamps to verify ownership before deletion.

Conclusion

Mutual exclusion and idempotency are pervasive challenges in distributed architectures. Distributed locks (Zookeeper, Redis, Tair, Cerberus) address the former, while GTIS provides a reliable pattern for the latter. Both rely on external storage, so their availability and performance directly affect system reliability.