How to Build a High‑Concurrency Leaderboard with Redis Zset and Sharding
This article explains how to design a scalable leaderboard using Redis Zset for moderate traffic and a sharded‑key plus local cache architecture for ultra‑high concurrency, detailing synchronization strategies, data flow, and practical implementation steps.
Leaderboards are common features in games, sales, and many other applications, helping to boost user enthusiasm and improve retention.
For typical traffic, a high‑concurrency leaderboard can be implemented with Redis Zset . The approach assumes that the top 10 users in any hour are among the top 200 overall. A periodic task (e.g., every 60 minutes) syncs the top 200 users from the database to a Redis Zset. Redis then maintains these 200 entries in real time, while updates to the database are not reflected until the next sync, providing near‑real‑time rankings.
When traffic spikes, this design may lag because hot data must wait for the next scheduled sync.
Ultra‑high concurrency solution
When data volume and request rates exceed a single Redis node’s limits (write bottleneck around 20 k ops, read bottleneck around 100 k ops), a single Zset key becomes a bottleneck. The solution is to shard the leaderboard across multiple small keys, each holding a portion of the data.
The sharding process splits the original Zset key into several keys, distributes the data, and each node maintains its own Zset. A periodic task collects the top 10 entries from each node, merges them to obtain the global top 10, and synchronizes the result to a local cache.
Using a local cache protects Redis nodes under extreme load; all requests hit the cache, while Redis updates the cache at regular intervals.
Summary
1) For high‑concurrency scenarios, a combination of periodic synchronization and Redis Zset efficiently serves a leaderboard.
2) For ultra‑high concurrency and massive data sets, sharding the Zset across multiple keys together with a local cache provides a scalable solution.
Lobster Programming
Sharing insights on technical analysis and exchange, making life better through technology.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.