Server-Side Caching: Local, Distributed, and Multi-Level Cache Architecture Practices

This article provides a comprehensive overview of server-side caching systems, covering cache positioning, key attributes, and implementation strategies.

Cache Positioning: Caching is a classic space-for-time trade-off to improve performance. The primary reasons for introducing caching are: (1) Alleviating CPU pressure by storing method results, pre-computing content, and reusing common data; (2) Alleviating I/O pressure by converting slow介质 (network, disk) access to fast memory access.

Cache Attributes: Four key dimensions are considered: Throughput (measured in OPS - Operations per Second), Hit Rate (ratio of successful cache hits to total requests), Extended Features (max capacity, expiration, statistics), and Distributed Support (in-process vs. distributed caching).

Local Caching: The article compares HashMap, Guava Cache, Ehcache, and Caffeine. Throughput analysis shows HashMap has highest throughput but lacks concurrency control; ConcurrentHashMap uses segment-based locking; Guava Cache uses synchronous processing with segment locking; Caffeine employs async log submission for better concurrency. Eviction strategies discussed include FIFO, LRU, LFU, TinyLFU, and W-TinyLFU (which combines LRU and LFU advantages using Window Cache and Main Cache with Segmented LRU).

Distributed Caching: Redis has become the de facto standard for distributed caching. Ehcache and Infinispan support both embedded and distributed deployment modes.

Multi-Level Cache: Transparent Multilevel Cache (TMC) combines in-process cache (L1) with distributed cache (L2) and DB (L3). The typical flow: read L1 first, then L2 on miss, then query source and populate both caches.

Cache Consistency: The article discusses Cache Aside pattern (most commonly used): (1) Miss: query DB and populate cache; (2) Hit: return from cache; (3) Update: write to DB first, then invalidate cache. Other patterns include Read Through, Write Through, and Write Behind. The Helios implementation uses an AP model approach: update DB, refresh distributed cache, then trigger local cache refresh via MQ.

Cache Monitoring: Uses sliding time window statistics with Disruptor for async event consumption. Sketch is used to filter low-frequency data and avoid stability risks.

Hot Key Detection: Local hot key detection is preferred over global detection for balanced traffic scenarios. The process involves: intercepting key access, performing frequency statistics using Sketch, hot key admission based on period-over-period comparison, and evicting non-hot keys during cycle rotation.