Understanding and Solving Redis Hot Key Problems

When building C‑side services, introducing a first‑level cache like Redis reduces database pressure but can create new challenges, notably the hot‑key problem where a large number of requests target the same key, overloading a single shard and potentially crashing the cluster.

Background : In a Redis cluster, keys are hashed to slots that map to specific master‑slave groups. During spikes such as flash‑sale events, many requests may hit the same key, concentrating traffic on one node, causing CPU, network, or memory bottlenecks—this is the hot‑key issue.

1. Hot‑Key Detection

1.1 Slot‑level QPS monitoring : Track QPS per slot and compare against others to spot uneven traffic. This method is simple but coarse and may miss precise hot‑key identification.

1.2 Proxy‑based statistics : If a proxy sits in front of the cluster, count requests per key within a sliding window at the proxy, flagging keys that exceed thresholds. This requires a proxy capable of such metrics.

1.3 Redis LFU hot‑key discovery : Redis 4.0+ provides an LFU‑based hot‑key detection; the command redis-cli --hotkeys lists hot keys on a node. It can be scheduled periodically.

1.4 Client‑side detection : Instrument Redis client code to count accesses per key using a sliding window; when a threshold is crossed, report to a central server. This adds memory overhead and may increase GC pressure in Java/Go.

2. Hot‑Key Mitigation

2.1 Rate limiting : Apply throttling to the offending key or slot. Effective for emergency stop‑gap but may impact business functionality.

2.2 Second‑level (local) cache : Add a local cache (e.g., GuavaCache in Java) in front of Redis to offload hot‑key reads. Example code:

// Local cache initialization and construction
private static LoadingCache<String, List<Object>> configCache = CacheBuilder.newBuilder()
    .concurrencyLevel(8) // set according to CPU cores
    .expireAfterWrite(10, TimeUnit.SECONDS) // TTL
    .initialCapacity(10)
    .maximumSize(10)
    .recordStats()
    .build(new CacheLoader<String, List<Object>>() {
        @Override
        public List<Object> load(String hotKey) throws Exception {
            // load logic here
            return null;
        }
    });

// Retrieve from local cache
Object result = configCache.get(key);

Local caching reduces Redis load but introduces data‑staleness proportional to the TTL.

2.3 Key sharding (splitting) : Replace a single hot key with multiple sub‑keys (e.g., good_100_copy1 … good_100_copy4 ) and distribute writes/reads across them, using a hash of the client IP/MAC or a random startup number to select the suffix.

2.4 Configuration‑center based local cache : Treat cache entries like configuration items. Services load all configs at startup, then long‑poll a central config service (e.g., Nacos) for updates, refreshing local memory instantly while keeping read latency low.

2.5 Pre‑emptive measures : For extreme scenarios (e.g., flash‑sale), isolate business domains, provision dedicated Redis clusters, or apply circuit‑breaker and disaster‑recovery tactics.

3. Integrated Solutions

Open‑source tools (e.g., JD’s hot‑key framework) combine client‑side detection, reporting, and server‑side distribution of hot‑key information to automatically switch to local caches with synchronized updates, offering a more complete, production‑ready approach.

Conclusion

The article outlines detection and mitigation techniques for Redis hot‑key problems, weighing trade‑offs such as consistency loss, implementation complexity, and infrastructure constraints, enabling teams to choose the most suitable strategy for their specific workload.