Effective Cache Strategies for Large Distributed Systems

In large distributed systems, especially high‑traffic services like Douyin, cache design is essential to avoid catastrophic load on the persistence layer and to keep response latency low.

Client‑Side Cache

Client‑side caching mainly relies on HTTP headers such as Cache‑Control: max‑age and conditional requests using Last‑Modified and ETag . Short‑lived client caches can reduce millions of daily requests by avoiding network round‑trips.

Typical Spring code for setting a max‑age looks like:

ResponseEntity.ok().cacheControl(CacheControl.maxAge(3, TimeUnit.SECONDS)).body()

Relevant imports:

import org.springframework.http.CacheControl;
import org.springframework.http.ResponseEntity;

Other directives such as no-store , no-cache , and must-revalidate control whether data is cached or must be revalidated before use.

Server‑Side Cache

CDN Cache

CDN caches static assets (images, audio, video) and can also cache infrequently changing dynamic data, reducing origin‑server load and latency.

Redis Cache

Redis is the most common distributed cache. Its five core data types—String, List, Set, Sorted Set, and Hash—cover >95% of use cases. Sorted Sets enable ranking (e.g., live‑stream leaderboards) while Sets support set operations such as intersection, union, and difference.

Bitmap can store billions of flags using only megabytes of memory, useful for sign‑in or presence tracking.

Typical Redis consistency challenges include cache‑database divergence, cache breakdown, penetration, and avalanche. Consistency is usually eventual; read‑only caches are common.

Consistency Patterns

Two common update flows are:

Update the database first, then delete the cache.

Delete the cache first, then update the database.

Both can fail under high concurrency; the recommended approach is to update the DB then delete the cache, optionally with retry logic or short TTLs.

Cache Breakdown, Penetration, Avalanche

Breakdown occurs when a hot key expires and a flood of requests hits the DB. Solutions include locking the first request or avoiding TTL for hot keys.

Penetration (requests for nonexistent keys) can be mitigated by caching null objects or using a Bloom filter.

Avalanche (many keys expiring simultaneously) is prevented by staggered TTLs and high‑availability Redis clusters.

Example of caching a null object in Java:

public class UserServiceImpl {
    @Autowired UserDAO userDAO;
    @Autowired RedisCache redisCache;

    public User findUser(Integer id) {
        Object obj = redisCache.get(id.toString());
        if (obj != null) {
            if (obj instanceof NullValueResultDO) return null;
            return (User) obj;
        } else {
            User user = userDAO.getUser(id);
            if (user != null) redisCache.put(id.toString(), user);
            else redisCache.put(id.toString(), new NullValueResultDO());
            return user;
        }
    }
}

Bloom filters provide high‑speed, low‑memory membership tests with a configurable false‑positive rate.

Network Timeouts & High‑Complexity Operations

Redis calls traverse the network, so timeouts can occur. Retrying failed calls and monitoring latency are essential. Operations on large collections (e.g., HGETALL , SMEMBERS ) have O(N) complexity; splitting big keys or sharding (using crc16(key) % 16384 ) mitigates the “big‑key” problem.

Message‑Queue Assisted Cache Updates

Publish‑subscribe patterns let producers emit change events; consumers update the cache asynchronously, ensuring eventual consistency without blocking the request path.

Scheduled Tasks & Local Cache

Periodic jobs can preload rarely‑changed data (e.g., whitelist/blacklist) into an in‑process map, reducing remote calls. In Spring Boot, enable scheduling with @EnableScheduling and annotate methods with @Scheduled(cron = "0 0 2 * * ?") .

Local Cache Annotations

Spring’s @Cacheable and @CachePut provide declarative caching. For more control, Google’s LoadingCache (Guava) allows custom expiration and asynchronous reloads:

private final LoadingCache
> tagCache = CacheBuilder.newBuilder()
    .concurrencyLevel(4)
    .expireAfterWrite(5, TimeUnit.SECONDS)
    .initialCapacity(8)
    .maximumSize(10000)
    .build(new CacheLoader
>() {
        @Override
        public List
load(String key) { return get(key); }
        @Override
        public ListenableFuture
> reload(String key, List
oldValue) {
            ListenableFutureTask
> task = ListenableFutureTask.create(() -> get(key));
            executorService.execute(task);
            return task;
        }
    });

Extending @Cacheable with TTL and auto‑refresh combines simplicity with flexibility.

Summary

Choosing the right cache—client, CDN, Redis, or local—involves trade‑offs among cost, consistency, and performance. Proper TTL settings, staggered expirations, locking, Bloom filters, message‑queue updates, and scheduled pre‑loading together form a robust caching strategy for billion‑user distributed applications.