Go Distributed ID Generation: UUID, ShortUUID, XID, KSUID, ULID, Snowflake, and Segment Modes

This article, originating from a recent discussion on Golang distributed ID generation, provides developers with practical experience and guidance on a variety of ID generation strategies.

1. Local ID generators – UUID

Two popular Go packages are mentioned:

github.com/google/uuid – supports only V1 and V4.

github.com/gofrs/uuid – supports all five UUID versions.

The five UUID versions differ as follows:

Version 1 – based on MAC address and timestamp.

Version 2 – based on timestamp, MAC address and POSIX UID/GID (DCE 1.1).

Version 3 – MD5 hash of a namespace and name.

Version 4 – pure random numbers.

Version 5 – SHA‑1 hash of a namespace and name.

Key characteristics: five selectable versions, fixed length of 36 characters, unordered.

package mian
import (
    "github.com/gofrs/uuid"
    "fmt"
)
func main() {
    // Version 1: time + MAC address
    id, err := uuid.NewV1()
    if err != nil {
        fmt.Printf("uuid NewUUID err:%+v", err)
    }
    fmt.Println("id:", id.String(), "length:", len(id.String()))

    // Version 4: pure random, panic on error internally
    id, err = uuid.NewV4()
    if err != nil {
        fmt.Printf("uuid NewUUID err:%+v", err)
    }
    fmt.Println("id:", id, "length:", len(id.String()))
}

2. shortuuid

shortuuid builds on UUID V4 but produces a shorter, fixed‑length (22 characters) identifier.

Features: shorter than UUID, fixed length, still globally unique.

package mian
import (
    "github.com/lithammer/shortuuid/v4"
    "fmt"
)
func main() {
    id := shortuuid.New()
    fmt.Println("id:", id, "length:", len(id))
    id = shortuuid.NewWithNamespace("http://127.0.0.1.com")
    fmt.Println("id:", id, "length:", len(id))
    str := "12345#$%^&*67890qwerty/;'~!@uiopasdfghjklzxcvbnm,.()_+·><"
    id = shortuuid.NewWithAlphabet(str)
    fmt.Println("id:", id, "length:", len(id))
}

3. xid

xid combines a timestamp, process ID, MAC address and a random component. Ordering is achieved by atomically incrementing the random part, which also mitigates clock‑rollback issues.

Characteristics: short length, ordered, non‑repeating, atomic increment avoids clock rollback.

package mian
import (
    "github.com/rs/xid"
    "fmt"
    "crypto/md5"
)
func main() {
    id := xid.New()
    containerName := "test"
    containerNameID := make([]byte, 3)
    hw := md5.New()
    hw.Write([]byte(containerName))
    copy(containerNameID, hw.Sum(nil))
    id[4] = containerNameID[0]
    id[5] = containerNameID[1]
    id[6] = containerNameID[2]
    fmt.Println("id:", id, "length:", len(id))
}

4. ksuid

KSUID consists of a 4‑byte timestamp followed by random bytes, yielding a 20‑character string.

package mian
import (
    "github.com/segmentio/ksuid"
    "fmt"
)
func main() {
    id := ksuid.New()
    fmt.Println("id:", id, "length:", len(id))
    id1 := ksuid.New()
    id2 := ksuid.New()
    fmt.Println(id1, id2)
    compareResult := ksuid.Compare(id1, id2)
    fmt.Println(compareResult) // 1
    isSorted := ksuid.IsSorted([]ksuid.KSUID{id2, id1})
    fmt.Println(isSorted) // true (descending)
}

5. ulid

ULID also mixes a timestamp with randomness, producing a 26‑character identifier.

package mian
import (
    "github.com/oklog/ulid"
    "fmt"
    "time"
    "math/rand"
)
func main() {
    t := time.Now().UTC()
    entropy := rand.New(rand.NewSource(t.UnixNano()))
    id := ulid.MustNew(ulid.Timestamp(t), entropy)
    fmt.Println("id:", id.String(), "length:", len(id.String()))
}

6. Snowflake

The classic Snowflake algorithm generates IDs from a timestamp, machine ID and sequence number. Compared with UUID, it does not expose MAC addresses and produces more compact IDs, but it inherits the clock‑rollback problem.

package main
import(
    "fmt"
    "github.com/bwmarrin/snowflake"
)
func main() {
  node, err := snowflake.NewNode(1)
  if err != nil {
    fmt.Println(err)
    return
  }
  id := node.Generate().String()
  fmt.Println("id:", id, "length:", len(id))
}

7. Database auto‑increment IDs

Traditional primary‑key auto‑increment IDs are simple, ordered, and space‑efficient (INT/BIGINT). However, they expose business volume, suffer from concurrency limits, and eventually require sharding when the range is exhausted.

8. Redis‑based ID generation

Using Redis atomic commands INCR/INCRBY yields higher performance and flexibility than DB auto‑increment, but introduces a single‑point dependency on Redis and possible network bottlenecks under heavy load.

9. ZooKeeper unique IDs

ZooKeeper’s Zxid provides a globally unique, monotonically increasing version number for each Znode change. While it solves ordering, it creates a strong dependency on ZooKeeper and can become a network bottleneck.

10. Segment (Leaf) mode

Instead of per‑row DB increments, a business tag (biz_tag) records the current maximum ID and step size. When a segment is exhausted, the service updates the max_id and obtains a new range of IDs.

Advantages:

Retains auto‑increment benefits while offering better performance.

Highly configurable and extensible.

Cache‑driven, allowing short‑term spikes to be handled smoothly.

Disadvantages:

IDs are ordered and can expose business volume, making them unsuitable for order numbers.

Cache refill latency may cause performance fluctuations.

Strong reliance on the underlying database.

11. Enhanced Leaf‑segment (double‑cache)

To mitigate cache‑refill latency, a background process pre‑fetches new segments once the current cache reaches a configurable usage threshold.

12. Tinyid

Tinyid follows the same segment principle, pre‑loading ID ranges for low‑latency generation.

13. Leaf‑snowflake and clock rollback handling

Clock rollback occurs when a server’s clock moves backwards, either due to manual adjustment or synchronization issues. The solution presented combines ZooKeeper’s sequential Zxid with a local cache, ensuring monotonic IDs even when the clock drifts. If the drift exceeds 5 ms, the system alerts and aborts startup.

14. Implementation diagram

The article includes architectural diagrams illustrating the integration of ZooKeeper caching within the Leaf‑snowflake design.

Author

Chen Dong – Backend development engineer at Tencent, focusing on middleware for Tencent Video, with extensive experience in message queues and Go performance optimization.

Recommended reading

C++ Asynchronous: asio coroutine implementation

Go Component: context study notes

The 6th Techo TVP Developer Summit & Tencent Cloud Big Data Summit

Is low‑code a new wave or just another hype?