Analysis and Solutions for Redis Distributed Lock Over‑selling in High‑Concurrency Seckill Scenario
This article examines a real‑world flash‑sale incident where Redis distributed locks failed, causing severe overselling, and presents root‑cause analysis, safer lock implementation with Lua scripts, atomic stock handling, refactored Java code, and deeper reflections on lock necessity and further optimizations.
Using Redis for distributed locks is common, but this article analyzes a real incident where a flash‑sale of a scarce product caused massive overselling despite the lock.
The project used a distributed lock for seckill orders; a promotion of 100 bottles of "Feitian Maotai" resulted in 200 bottles sold, classified as a P0 incident.
Incident Scene
Investigation revealed that the lock expired while user‑service verification took longer than 10 seconds, allowing subsequent requests to acquire the lock and later release it, breaking the lock’s one‑to‑one mapping.
Root Causes
No fault‑tolerance for dependent services.
Distributed lock not truly safe – lock can be released by a different thread.
Non‑atomic stock check.
These issues combined caused the oversell.
Solutions
Safer Distributed Lock
Implement lock release with a Lua script that checks the stored value before deleting.
public void safedUnLock(String key, String val) {
String luaScript = "local in = ARGV[1] local curr=redis.call('get', KEYS[1]) if in==curr then redis.call('del', KEYS[1]) end return 'OK'";
RedisScript
redisScript = RedisScript.of(luaScript);
redisTemplate.execute(redisScript, Collections.singletonList(key), Collections.singleton(val));
}Atomic Stock Check
Leverage Redis’s atomic increment/decrement to adjust stock safely.
// redis will return the result atomically
Long currStock = redisTemplate.opsForHash().increment("key", "stock", -1);Refactored Code
Introduce a DistributedLocker class and use UUID values for lock ownership, combined with atomic stock decrement.
public SeckillActivityRequestVO seckillHandle(SeckillActivityRequestVO request) {
SeckillActivityRequestVO response;
String key = "key:" + request.getSeckillId();
String val = UUID.randomUUID().toString();
try {
Boolean lockFlag = distributedLocker.lock(key, val, 10, TimeUnit.SECONDS);
if (!lockFlag) {
// business exception
}
// user activity validation omitted
Long currStock = stringRedisTemplate.opsForHash().increment(key + ":info", "stock", -1);
if (currStock < 0) {
log.error("[抢购下单] 无库存");
// business exception
} else {
// generate order, publish event, build response
}
} finally {
distributedLocker.safedUnLock(key, val);
}
return response;
}Deep Reflection
Is a Distributed Lock Necessary?
While atomic stock decrement can avoid overselling, the lock helps throttle traffic and protect downstream services from excessive load.
Lock Selection
RedLock offers higher reliability at the cost of performance; choose based on scenario requirements.
Further Optimizations
Cache stock per server, use hash‑based routing, and employ ConcurrentHashMap for fast in‑memory checks, potentially eliminating the need for Redis in some cases.
Conclusion
Overselling scarce items is a severe incident; thorough design, atomic operations, and proper lock handling are essential for reliable high‑concurrency systems.
Java Captain
Focused on Java technologies: SSM, the Spring ecosystem, microservices, MySQL, MyCat, clustering, distributed systems, middleware, Linux, networking, multithreading; occasionally covers DevOps tools like Jenkins, Nexus, Docker, ELK; shares practical tech insights and is dedicated to full‑stack Java development.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.