Hot Data Detection and Optimized Caching with Caffeine and HotCaffeine

1. What is Hot Data?

Hot data refers to data that is frequently accessed within a certain time window. Examples include permanently hot videos (e.g., a popular TV series during its broadcast), periodic hot data (e.g., European Cup matches), and bursty hot data (e.g., earthquake reports or celebrity live streams).

Caching hot data can significantly improve application performance.

2. Trade‑offs

Different hot‑data categories require different caching strategies: permanently hot data should be fully cached, bursty hot data needs effective detection mechanisms, and periodic hot data can be treated as bursty within each cycle. However, caching everything in memory is impractical due to GC pauses and the large size of video datasets.

Cache hit rate is directly related to memory size: larger memory allows more cached items and higher hit rates, while smaller memory reduces both.

3. Best Hit Rate with Caffeine

When access probability is constant, LFU (Least Frequently Used) yields the highest hit rate but suffers from inability to adapt to changing frequencies and high memory overhead. LRU adapts to time changes but cannot achieve the highest hit rate.

Caffeine uses an improved TinyLFU algorithm, which combines a Bloom‑filter‑based Doorkeeper to filter low‑frequency items and a Count‑Min Sketch for lightweight frequency counting.

W‑TinyLFU adds a small Window cache (1% of total) using LRU; new items first enter the Window, and when it fills, items are evicted through TinyLFU, ensuring bursty hot items have a chance to stay.

4. Hotspot Detection

A centralized hotspot detection system aggregates client‑side access counters via a ring buffer, reports them periodically, and uses a sliding‑window algorithm to identify high‑frequency keys, notifying client caches in real time.

The detection pipeline includes client counting, rotation, pushing to a detector queue, parallel consumption, sliding‑window frequency calculation, and notification to Caffeine caches.

5. HotCaffeine

HotCaffeine builds on JD.com's hotkey project, adding multi‑cache configuration, flexible hotspot rule definitions, real‑time hot‑key visualization, call‑volume distribution analysis, Top‑K hot‑key metrics, and throughput optimizations that raise processing capacity from 350k to 590k keys/second.

Its architecture consists of etcd for configuration, client SDK for data collection, worker for aggregation and hot‑key computation, and a dashboard for management.

HotCaffeine is open‑source on GitHub: https://github.com/sohutv/hotcaffeine.

6. References

Key references include the Caffeine library, detailed Caffeine tutorials, the W‑TinyLFU paper, Segment LRU research, studies on hot data in object storage systems, and the original hotkey project.