Ensuring Idempotency with Unique IDs: UUID, Snowflake, and Distributed ID Generation

Why Ensure Idempotency

In programming, “idempotency” means that executing an operation any number of times has the same effect as executing it once. An idempotent interface guarantees the same result whether it is called once or repeatedly, which is crucial in scenarios such as user order placement.

Service calls can end in success, failure, or timeout. A timeout leaves the downstream service’s state unknown, and upstream services may retry, raising concerns about duplicate orders, double‑deducted inventory, or charging the user twice.

Will two identical orders be created?

Will inventory be deducted twice?

Will the user be charged twice?

If each order carries a unique sequence number, the order service can record the status of that operation. When the status is successful, subsequent executions can be intercepted, preventing the above problems. This approach places the responsibility for idempotency on the downstream service.

Additionally, the order service can expose an order‑status query API so that the upstream service checks whether an order has already been paid before retrying.

Generally, a service should ensure its own idempotency rather than delegating it to callers.

Unique ID

To achieve idempotency, a unique ID must mark each transaction. There are two main ways to obtain such an ID:

A centralized ID‑allocation service.

The upstream service generates the ID itself, provided it guarantees no collisions.

Using a centralized allocator (e.g., MySQL auto‑increment) adds an extra request per call. For high‑performance requirements, generating the ID locally is preferable. Common generation methods include:

UUID

UUID (Universally Unique Identifier) is a 128‑bit value. Although not absolutely collision‑free, the probability of duplication is negligible. UUIDs are generated from components such as MAC address, timestamp, namespace, and random numbers, making them opaque, large, and non‑sequential.

128 bits consume considerable storage.

Not human‑readable.

Cannot guarantee monotonic ordering.

Online generators are available for experimentation.

Snowflake

Twitter’s open‑source Snowflake algorithm produces a long‑type unique ID with three parts:

41 bits for timestamp in milliseconds (≈69.7 years).

10 bits for machine identifier (up to 1024 nodes).

12 bits for a sequence number within the same millisecond (up to 4096 IDs per ms).

Implementations exist in many languages; the core idea is similar to Redis or MongoDB global ID generators.

Redis‑based distributed ID generator example: github.com/hengyunabc/redis-id-generator

Its key concepts are:

41 bits for time (millisecond precision), usable for 41 years.

12 bits for logical shard ID (max 4095).

10 bits for auto‑incrementing ID, allowing up to 1024 IDs per node per millisecond.

Shared Storage

When the idempotency service is distributed, the unique ID store must be shared so each instance remains stateless. Options include MySQL or key‑value stores such as Redis; the author uses Redis for storing unique keys.

Avoid Unnecessary Queries

Most requests are not duplicates. Querying the unique ID store for every request can waste resources. With MySQL, an INSERT ... ON DUPLICATE KEY UPDATE statement can determine whether the key already exists; with Redis, SETEX succeeds only if the key is absent.

insert into ... values ... on DUPLICATE KEY UPDATE ...