Distributed ID Generation Using Redis and Snowflake Algorithm

This article introduces several existing global ID generation schemes and proposes a deployment-free ID generation solution based on mainstream server services. The article is structured into seven sections covering technical background, existing solutions, online business server conditions, simple schemes, registration mechanisms, and an ideal implementation.

The technical background explains that MySQL and Redis have become essential components in modern backend services, playing crucial roles in data persistence and fast processing. With the development of distributed services, the demand for global ID generation services has become increasingly strong.

The article reviews existing solutions including using MySQL's auto-increment feature and Twitter's Snowflake algorithm. The MySQL approach involves creating a table with an auto-increment ID column and performing transactional operations to retrieve IDs. However, this method is slow due to the need for database write and read operations for each ID.

The Snowflake algorithm is presented as a 64-bit ID generation algorithm that divides bits into different segments: a sign bit, timestamp, machine ID, and sequence number. This algorithm offers high generation efficiency but requires separate deployment.

Considering that business servers typically handle complex business processing with limited QPS capacity (usually under 2000-3000), the article proposes a solution where each business server generates its own IDs using a simplified Snowflake algorithm, eliminating the need for separate ID generation services.

The proposed solution uses Redis as the storage system for the ID generation algorithm. The simplest approach involves using a common Redis key with increment operations, combining the resulting number with a timestamp to create a complete ID. However, this approach has scalability issues as traffic increases.

To address scalability, the article introduces a registration mechanism where different machines are assigned unique numbers and Redis keys to distribute Redis operation pressure. The registration process uses machine domain names as identifiers, which are globally unique and reliable.

The registration mechanism involves several steps: obtaining the machine name using gethostname(), adding the machine to a registration list using sadd operation, incrementing a machine count using incr operation, and storing the machine domain name to number mapping using set with NX option.

The ideal scheme structures Redis keys as autogid:idcount:<time>:<machineid>, where the key changes with time and machine number. Each second, each machine uses a different key. The ID structure combines timestamp, machine ID, and sequence number using bit shifting operations.

The article provides detailed explanations of the ID generation process, including handling cases where the number of IDs generated in a second exceeds the maximum capacity, and special considerations for key expiration and batch generation.

The proposed solution offers a practical approach to distributed ID generation that leverages existing infrastructure, provides good scalability, and maintains compatibility with the Snowflake algorithm for future upgrades.