Cache Basics, Types, Patterns, and Common Issues

1. Why Use Cache

Caching can be divided into local cache and distributed cache. In Java, using a built‑in Map or Guava implements a local cache that is lightweight, fast, and lives as long as the JVM, but each instance maintains its own copy, so consistency is not guaranteed. Distributed caches such as Redis or Memcached share a single data store across instances, providing consistency at the cost of higher architectural complexity and the need for high availability.

2. Cache Classification

2.1 Local Cache

Only the current instance can use it; in clustered deployments local caches cannot be shared and consume the instance's memory. Java options include Map, Guava, and Caffeine.

Local cache comparison:

1. Using Map or ConcurrentHashMap requires you to design eviction policies yourself.

2. Guava builds on ConcurrentHashMap and provides built‑in eviction, monitoring of load/hit rates, and thread‑safety.

3. Caffeine supports asynchronous loading with CompletableFuture, uses both LRU and LFU policies, and is the default cache in Spring 5.

Summary : Guava is better than raw Java containers, but Caffeine improves on Guava and is currently the best choice for a local cache framework.

2.2 Distributed Cache

Shared across services in a cluster and does not consume application memory; examples are Redis and Memcached.

Redis vs. Memcached

Redis usage recommendations :

Avoid large keys (value > 10 KB).

Avoid setting massive expiration times simultaneously.

Implement cache‑penetration protection in the DB layer.

Avoid key update operations.

Use MGET/pipeline for batch reads and avoid heavy computation inside pipelines.

High Availability (Redis)

Redis provides Sentinel mode (master‑slave with failover) and Cluster mode (sharded multi‑master). Sentinel cannot improve write throughput and incurs a brief outage during failover, while Cluster offers linear scalability and better fault tolerance.

3. Cache Reading

The recommended practice is the Cache‑Aside pattern: always read from the cache first (cache hit); if missing, read from the database (cache miss), then populate the cache for subsequent reads.

4. Common Cache Problems

4.1 Data Inconsistency Between DB and Cache

When both DB and cache are written, consistency issues arise. The common solution is to delete the cache after a DB update, ensuring the next read repopulates fresh data.

Solution Details

4.1.1 Delete Cache After DB Update

Deleting rather than updating avoids complex recomputation and transaction rollback problems.

4.1.2 Use Binlog Monitoring (e.g., Canal, Databus)

Listen to DB changes and invalidate or update related cache entries accordingly.

4.2 Cache Penetration

Occurs when requests for non‑existent keys repeatedly miss both cache and DB.

Mitigations

4.2.1 Cache Empty Values

Store a placeholder for missing keys to prevent repeated DB hits.

4.2.2 Bloom Filter

Use a Bloom filter to quickly reject requests for keys that definitely do not exist.

4.3 Cache Breakdown

When a hot key expires, a surge of requests hits the DB.

Solutions

Proactive refresh via scheduled tasks.

Update expiration time on each access.

Multi‑level caching (e.g., L1 short‑TTL, L2 long‑TTL).

Mutex lock to ensure only one request repopulates the cache.

public User queryById(int id) throws InterruptedException {
    User user = (User) redisTemplate.opsForValue().get(id + "");
    if (user == null) {
        if (tryLock(id + "")) {
            try {
                System.out.println(Thread.currentThread().getName() + " got lock, query DB");
                user = deptDao.queryById(id);
                if (user == null) {
                    // prevent penetration
                    redisTemplate.opsForValue().set(id + "", new User(), 30, TimeUnit.MINUTES);
                } else {
                    redisTemplate.opsForValue().set(id + "", user);
                }
            } finally {
                unlock(id + "");
            }
        } else {
            user = (User) redisTemplate.opsForValue().get(id + "");
            if (user == null) {
                System.out.println(Thread.currentThread().getName() + " waiting");
                Thread.sleep(100);
                return queryById(id);
            }
        }
    }
    return user;
}

4.4 Cache Avalanche

When many keys expire simultaneously, the DB is overwhelmed.

Mitigation Steps

Add random jitter to expiration times.

Ensure high availability (Redis Sentinel or Cluster).

Use local caches (Caffeine) and rate‑limiting/fault‑tolerance mechanisms (e.g., Hystrix).

Enable persistence for faster recovery.

4.5 Clustered Local Cache Updates

In multi‑instance deployments, use a Redis queue or external message queue to broadcast cache invalidations so all instances update their local caches consistently.

4.6 Hot Keys

What Is a Hot Key?

Keys that are accessed extremely frequently, either in databases (e.g., MySQL) or in Redis.

Hot‑Key Solutions

Detect hot keys quickly and move them into JVM memory to avoid costly remote accesses. Real‑time detection (within seconds) is essential.

Key Detection Metrics

Real‑time: detect within ~1 second.

Accuracy: avoid false positives/negatives.

Cluster consistency: ensure deletions propagate across all nodes.

JdHotKey

JdHotKey is an open‑source millisecond‑level hot‑key detection framework from JD.com, capable of handling hundreds of thousands of QPS and automatically promoting hot keys into JVM memory.

5. Summary

Using caches increases system complexity; choose between local and distributed caches based on business needs, set appropriate eviction and expiration policies, and apply the discussed techniques to mitigate risks such as inconsistency, penetration, breakdown, avalanche, and hot‑key overload.