Distributed ID Generation Strategies: UUID, Auto‑Increment, Segment Mode, Redis INCR, Snowflake and Open‑Source Implementations

Table of Contents

Background

1. UUID

2. Database Auto‑Increment 2.1 Primary‑Key Table 2.2 Auto‑Increment Step

3. Segment Mode

4. Redis INCR

5. Snowflake Algorithm

6. Meituan (Leaf)

7. Baidu (Uidgenerator)

8. Didi (TinyID)

Summary Comparison

Background

In complex distributed systems, a large amount of data often needs a globally unique identifier. Simple auto‑increment IDs cannot guarantee uniqueness across sharded tables, and different scenarios impose requirements such as monotonic increase, trend increase, and security against sequential ID exposure.

Trend increase: Use ordered primary keys to maintain B‑tree index performance.

Monotonic increase: Ensure each new ID is larger than the previous one for sorting needs.

Security: Avoid predictable sequential IDs that could reveal business volume.

The article reviews several distributed ID solutions, discussing their advantages, disadvantages, suitable scenarios, and provides code samples.

1. UUID

UUID ( Universally Unique Identifier ) is generated from timestamp, counter and hardware MAC address, yielding a 36‑character string (8‑4‑4‑4‑12). Java provides a built‑in generator:

import java.util.UUID;

public class Test {
    public static void main(String[] args) {
        System.out.println(UUID.randomUUID());
    }
}

Example output: b0378f6a-eeb7-4779-bffe-2a9f3bc76380

Although UUID guarantees global uniqueness, it is rarely used in practice because of high storage cost, potential security leaks (MAC address exposure), and poor performance as a MySQL primary key due to its length and randomness.

2. Database Auto‑Increment ID

MySQL's auto‑increment can provide unique IDs within a single table, but not across multiple shards. Two common remedies are:

2.1 Primary‑Key Table

Create a dedicated table that only stores the next ID and retrieve it via an atomic REPLACE operation:

CREATE TABLE `unique_id` (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `biz` char(1) NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `biz` (`biz`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;

BEGIN;
REPLACE INTO unique_id (biz) VALUES ('o');
SELECT LAST_INSERT_ID();
COMMIT;

2.2 Auto‑Increment Step

Set different auto‑increment step values for each MySQL instance so that IDs generated on different nodes do not clash. Example: instance 1 uses step 1, instance 2 uses step 2. The step can be viewed with:

SHOW VARIABLES LIKE '%increment%';

This method may encounter scalability issues under high concurrency.

3. Segment Mode

Segment mode fetches a range of IDs from the database and caches it in memory. When the cached range is exhausted, a new segment is requested using an optimistic‑lock version field. Example table:

CREATE TABLE id_generator (
  id int NOT NULL,
  max_id bigint NOT NULL COMMENT 'current max id',
  step int NOT NULL COMMENT 'segment length',
  biz_type int NOT NULL COMMENT 'business type',
  version int NOT NULL COMMENT 'optimistic lock version',
  PRIMARY KEY (`id`)
);

This reduces database pressure but suffers from discontinuity if a node crashes.

4. Redis INCR

Redis's atomic INCR command can generate a global counter:

@Component
public class RedisDistributedId {
    @Autowired
    private StringRedisTemplate redisTemplate;
    private static final long BEGIN_TIMESTAMP = 1659312000L;

    public long nextId(String item) {
        LocalDateTime now = LocalDateTime.now();
        long nowSecond = now.toEpochSecond(ZoneOffset.UTC);
        long timestamp = nowSecond - BEGIN_TIMESTAMP;
        String date = now.format(DateTimeFormatter.ofPattern("yyyy:MM:dd"));
        Long increment = redisTemplate.opsForValue().increment("id:" + item + ":" + date);
        return timestamp << 32 | increment;
    }
}

Redis‑based IDs face persistence challenges if the Redis server crashes.

5. Snowflake Algorithm

Twitter's Snowflake splits a 64‑bit integer into sign, timestamp (41 bits), datacenter ID (5 bits), worker ID (5 bits) and sequence (12 bits). It can generate up to 4096 IDs per millisecond per node. A Java implementation:

public class SnowflakeIdWorker {
    private final long twepoch = 1604374294980L;
    private final long workerIdBits = 5L;
    private final long datacenterIdBits = 5L;
    private final long maxWorkerId = -1L ^ (-1L << workerIdBits);
    private final long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);
    private final long sequenceBits = 12L;
    private final long workerIdShift = sequenceBits;
    private final long datacenterIdShift = sequenceBits + workerIdBits;
    private final long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
    private final long sequenceMask = -1L ^ (-1L << sequenceBits);
    private long workerId;
    private long datacenterId;
    private long sequence = 0L;
    private long lastTimestamp = -1L;
    // constructors, nextId(), tilNextMillis(), timeGen() omitted for brevity
    public static String getSnowId() {
        SnowflakeIdWorker sf = new SnowflakeIdWorker();
        return String.valueOf(sf.nextId());
    }
}

If the system clock moves backwards, Snowflake throws an exception to avoid duplicate IDs.

6. Meituan (Leaf)

Leaf is an open‑source project from Meituan that supports both segment mode and Snowflake mode. Snowflake mode relies on ZooKeeper to allocate sequential work IDs, while segment mode batches auto‑increment IDs from the database.

7. Baidu (Uidgenerator)

Uidgenerator is Baidu's Java implementation based on Snowflake, offering higher QPS (over 6 million per instance) and using MySQL to allocate worker IDs. Its timestamp field is only 28 bits, limiting the usable lifespan to about 8.5 years unless the bit allocation is adjusted.

8. Didi (TinyID)

TinyID builds on Meituan's Leaf‑segment algorithm, adding multi‑master support and a client library. It only supports segment mode (no Snowflake) and provides both HTTP and client‑side APIs.

Summary Comparison

The article concludes with a comparative table (omitted here) summarizing the trade‑offs of each solution regarding uniqueness, ordering, performance, scalability, and operational complexity.