Cache Strategies and Framework Selection for High‑Performance Systems

According to the current popular Internet layered architecture model, the simplest architecture is the Web response layer plus the DB storage layer. It evolved from a single‑machine mixed deployment of Web and DB, to separating them onto different physical machines to avoid hardware bottlenecks, then to Web clusters and DB master‑slave setups for high availability and read‑write separation. These upgrades aim at higher performance and lower latency.

Gaode, a nationwide navigation app, manages route data centrally. To keep data fresh, the data center continuously collects real‑world road changes and stores them in a database. At the same time, many internal services generate massive queries for the latest data, requiring control of DB connection counts, reduced response time, and real‑time data returns.

Balancing data freshness and performance, Gaode adopts different strategies for different business scenarios, achieving low‑latency data changes and cache synchronization while ensuring system stability.

The cache technologies mentioned are another powerful tool for performance improvement. However, no technology is universally best; the choice must fit the specific problem.

1. Why Use Cache

When a service is exposed to the outside, its stability is affected by CPU, memory, and I/O resources. For computations that are expensive or data that is frequently read from disk, caching the results avoids repeated resource consumption.

1.1 CPU Bottleneck

If a project performs many regular‑expression calculations or merges intermediate results many times, CPU usage stays high. Caching the results and retrieving them by key reduces CPU load.

1.2 I/O Bottleneck

Disk I/O is limited by rotation speed, seek time, and buffer size. Since CPU cache > memory > disk speed, when disk IOPS cannot be improved, data is cached in memory to relieve DB pressure.

2. Choosing Between Local and Distributed Cache

When cache is used to boost system throughput, the next question is whether to use local cache or distributed cache, and when multi‑level caching is needed.

2.1 Local Cache Pros and Cons

High stability because cache and application run in the same process.

No network overhead; data is accessed directly from the process memory.

Cons:

Shares JVM heap with the application, causing memory contention, no horizontal scaling, and possible frequent GC.

No persistence; data is lost on restart, requiring warm‑up for hot data.

Duplicate caches across processes waste memory.

Data consistency is hard when the same data is stored on multiple machines.

Local cache is suitable for data that is frequently accessed, rarely changes, and whose size can be estimated in advance.

2.2 Distributed Cache Pros and Cons

Centralized storage eliminates data redundancy, eases consistency management, and supports horizontal scaling.

Cache middleware provides unified management.

Cons:

Reliance on middleware stability; a failure may cause a cache avalanche.

Network overhead and serialization/deserialization cost.

Distributed cache is preferred for data that is frequently accessed but changes often, because it naturally guarantees consistency.

3. Selecting a Cache Framework

For local cache, simple scenarios can use HashMap (no contention) or ConcurrentHashMap (thread‑safe). Both lack expiration and eviction mechanisms.

When eviction is needed, Caffeine is recommended for its rich API and automatic expiration.

For distributed cache, the two mainstream choices are Redis and Memcached. Redis offers richer data structures, persistence, and compact memory usage, making it the default recommendation for most cases.

Summary: If no eviction algorithm is needed, choose ConcurrentHashMap ; if eviction and richer APIs are required, choose Caffeine .

Memcached uses fixed‑size memory slots, which can waste space; Redis allocates memory on demand, which is more compact but may cause fragmentation.

Summary: Redis’s rich data structures and persistence make it the preferred distributed cache.

4. Cache Expiration Strategies

Time‑based strategies:

Timed deletion: each key has a timer; precise but high CPU/memory cost.

Lazy deletion: delete on next access; CPU‑friendly but may retain expired data.

Periodic scanning: a single timer scans for expired entries at intervals, balancing CPU and memory usage.

Space‑based strategies (FIFO, LRU, LFU):

a) FIFO (First‑In‑First‑Out)

// The head of the linked list is the least recently accessed element to be removed
public class FIFOMap<K, V> extends LinkedHashMap<K, V> {
    private int maxSize;

    // LinkedHashMap inserts at tail; accessOrder=false means access does not move entries
    public FIFOMap(int maxSize) {
        super(maxSize, 0.75f, false);
        this.maxSize = maxSize;
    }

    // Called after each put/putAll; when true, the eldest entry (head) is removed
    @Override
    protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
        return size() > maxSize;
    }
}

b) LRU (Least Recently Used)

// The head of the linked list is the least recently used element to be removed
public class LRUMap<K, V> extends LinkedHashMap<K, V> {
    private int maxSize;

    // accessOrder=true moves accessed entries to the tail
    public LRUMap(int maxSize) {
        super(maxSize, 0.75f, true);
        this.maxSize = maxSize;
    }

    @Override
    protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
        return size() >= maxSize;
    }
}

c) LFU (Least Frequently Used)

Implemented in Redis via a sorted set (Zset) where the score is the access count (or a combination of timestamp and count). Elements with the smallest score are evicted.

5. Factors Affecting Cache Hit Rate

5.1 Cache Penetration – queries for keys that do not exist in DB. Solutions:

Cache null values (may waste memory if many).

Bloom filter – probabilistic structure to quickly filter non‑existent keys.

5.2 Cache Breakdown – when a hot key expires and many requests hit DB simultaneously. Solutions:

Randomized expiration times.

Proactive cache refresh before expiration.

5.3 Cache Avalanche – massive simultaneous expiration or middleware failure. Mitigation: random TTLs and health monitoring.

6. Data Consistency Strategies

Read operations: read cache first, on miss read DB and set cache.

Write operations: two main patterns.

DB first, then delete/update cache – risk of stale cache if cache update fails.

Delete/update cache first, then operate DB – if cache delete fails, the whole transaction aborts; if DB fails after cache delete, a cache miss occurs and old data is read later, which is acceptable.

Conclusion: deleting the cache before the DB operation provides the best atomicity with minimal business impact.

7. Summary

There is no universal golden standard for cache usage; the choice depends on business scenarios. Generally:

For local cache with predictable size, use JDK HashMap or ConcurrentHashMap .

For local cache requiring expiration or auto‑refresh, use Caffeine .

For distributed cache with rich data structures, use Redis .

Related articles: Coding Standards – Elegant Java Functions (Part 1) , Elegant Java Functions (Part 2) , Map Data Processing – Road Matching .