Understanding Cache Usage and Enterprise Cases: Types, Problems, and Architecture

Cache is essential for bridging performance gaps between fast processors and slower storage, and for improving user experience; failures can cause severe service disruptions, as illustrated by the 2015 Ctrip outage that cost millions of dollars.

When the MySQL buffer pool is insufficient, disk I/O blocks threads, leading to request queuing, timeouts, and potential circuit‑breaker activation, especially under high concurrency.

Cache types include client‑side cache, CPU cache (L1/L2/L3), single‑machine caches such as Ehcache and Guava, database query caches, distributed caches like Memcached and Redis, and network caches (CDN, Nginx proxy store).

Common high‑traffic cache problems are:

Cache penetration – queries for non‑existent data repeatedly hit the database; solutions are empty‑value caching or Bloom filters.

Cache breakdown – hot keys expire, flooding the database; solutions include never‑expiring hot keys with logical expiration or single‑threaded queueing.

Cache avalanche – many keys expire simultaneously; solutions are staggered TTLs, logical expiration, and high‑availability clustering.

Data drift – node failures in consistent‑hash clusters cause temporary data loss; mitigated by virtual nodes and balanced hashing.

Cache stampede – sudden spikes on uncached data cause overload; mitigated with locks or Promise‑based proxy patterns.

Cache pollution – unrelated workloads evict useful entries; mitigated by separating workloads and prioritizing access patterns.

Hot‑key handling – detection via monitoring, multi‑level caching, LRU, pre‑warming, and sharding.

Weibo’s large‑scale cache architecture evolved to handle billions of requests, employing multi‑layer caches, HA layers instead of consistent hashing, and a CacheService that abstracts proxy logic, async event handling, routing, and LRU optimization.

Redis is heavily customized at Weibo for persistence (bgsave, AOF+POS), memory efficiency (longset), and counting optimizations, demonstrating deep integration of a distributed cache into core services.

Zhihu’s homepage read‑filter cache mirrors CPU multi‑level cache design, using cache interception, replica expansion, and compression to achieve low latency and stability.

Overall, the article argues that caching is indispensable for backend systems, outlines typical pitfalls, and provides concrete architectural examples to guide robust cache implementation.