Distributed ID Generation: Requirements and Common Implementation Schemes

Why Auto‑Increment IDs Fail in Distributed Systems

In monolithic applications primary‑key auto‑increment is convenient, but after sharding databases for high‑traffic services (e.g., order tables) the same numeric ID can appear in different shards, breaking business logic.

Business Requirements for Distributed IDs

Global Uniqueness : No duplicate IDs across the whole system.

Trend Increase : Ordered IDs improve write performance for InnoDB’s clustered index.

Monotonic Increase : Guarantees that a later ID is larger than an earlier one, useful for versioning and sorting.

Security : IDs should not be strictly sequential to prevent easy data scraping.

Distributed ID Generation Schemes

UUID

Database Auto‑Increment

Segment Mode

Redis Implementation

Snowflake Algorithm

Baidu UidGenerator

Meituan Leaf

Didi TinyID

UUID

UUID (Universally Unique Identifier) is a 128‑bit value represented as 36 characters (8‑4‑4‑4‑12). Example: 863e254b-ae34-4371-87da-204b71d46a7b . The total number of possible UUIDs is 2^128 ≈ 3.4×10^38 .

Advantages

Very high performance; generated locally without network dependency.

Disadvantages

Large storage (16 bytes, 36‑character string).

Potential security risk if generated from MAC address.

Unordered nature can cause index fragmentation.

Database Auto‑Increment

MySQL’s auto_increment_increment and auto_increment_offset can be configured per server to generate distinct ranges. For two servers, set step=2, Server 1 starts at 1 (1,3,5,…), Server 2 starts at 2 (2,4,6,…). Expanding the cluster requires re‑configuring step and offsets, which can be cumbersome.

Drawbacks

IDs lose strict monotonicity, only trend‑increase.

Database becomes a bottleneck; each ID request hits the DB.

Segment Mode

The application fetches a range (e.g., [1,1000] ) from a table and uses the numbers in memory until exhausted. Table schema example:

CREATE TABLE id_generator (
  id int NOT NULL,
  max_id bigint(20) NOT NULL COMMENT 'current max id',
  step int(20) NOT NULL COMMENT 'segment length',
  biz_type int(20) NOT NULL COMMENT 'business type',
  version int(20) NOT NULL COMMENT 'optimistic lock version',
  PRIMARY KEY (id)
);

Key fields:

biz_type : distinguishes different business lines.

max_id : current maximum ID in the segment.

step : length of the segment.

version : optimistic‑lock version.

After using a segment, the application updates max_id and version in the DB.

Pros : Mature solutions (e.g., Baidu UidGenerator, Meituan Leaf).

Cons : Still depends on the database.

Redis Implementation

Redis provides atomic INCR / INCRBY commands. Because Redis is single‑threaded, IDs are unique and ordered. High concurrency may require a Redis cluster, which introduces similar sharding and step‑configuration challenges.

Advantages

Good performance, guarantees uniqueness and order.

Disadvantages

Requires adding Redis to the infrastructure.

Snowflake Algorithm

Twitter’s Snowflake splits a 64‑bit integer into four parts: 1‑bit sign (always 0), 41‑bit timestamp (≈69 years), 10‑bit machine ID (up to 1024 nodes), and 12‑bit sequence (up to 4096 IDs per millisecond). IDs are trend‑increasing and do not rely on external storage.

Advantages

High throughput, no third‑party dependency, flexible bit allocation.

Disadvantages

Clock rollback can cause duplicate IDs or service outage.

Baidu UidGenerator

Open‑source Java implementation based on Snowflake with enhancements: uses future timestamps to avoid sequence contention, employs a RingBuffer to cache IDs, and mitigates false sharing. Single‑machine QPS can reach 6 million.

Meituan Leaf

Leaf offers two modes: segment (Leaf‑segment) and Snowflake (Leaf‑snowflake). The segment table structure:

+------------+--------------+------+-----+-------------------+-----------------------------+
| Field      | Type         | Null | Key | Default           | Extra                       |
+------------+--------------+------+-----+-------------------+-----------------------------+
| biz_tag    | varchar(128) | NO   | PRI |                   |                             |
| max_id     | bigint(20)   | NO   |     | 1                 |                             |
| step       | int(11)      | NO   |     | NULL              |                             |
| desc       | varchar(256) | YES  |     | NULL              |                             |
| update_time| timestamp    | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+------------+--------------+------+-----+-------------------+-----------------------------+

Leaf‑segment batches ID ranges to reduce DB load; Leaf‑snowflake uses Zookeeper to assign worker IDs automatically.

Didi TinyID

TinyID is a Java‑based distributed ID system built on the database segment algorithm, extending Leaf‑segment with multi‑DB support and a client library. It offers easy integration but still depends on DB stability.

Pros

Convenient integration, mature solution.

Cons

Relies on database availability; clustering needed for high reliability.

Source: blog.csdn.net/u014427391/article/details/127408363