Design and Selection of Local In-Memory Cache Solutions for Two-Level Cache Architecture

Background

In high‑performance service architecture, cache is essential. Typically hot data is stored in remote caches like Redis or Memcached, falling back to the database on miss, which improves latency and reduces DB load.

As systems evolve, a single remote cache may be insufficient; combining a local cache (e.g., Guava Cache or Caffeine) as a first‑level cache with a remote cache as second‑level yields a two‑level cache architecture.

Why Use Local Cache

Local cache resides in process memory, offering extremely fast access for low‑frequency‑change data.

It reduces network I/O by avoiding remote cache calls.

Required Features of a Local Cache

Store, read, write operations.

Atomic/thread‑safe operations (e.g., ConcurrentHashMap).

Maximum size limit.

Eviction policies such as LRU or LFU.

Expiration strategies (time‑based, lazy, periodic).

Persistence.

Metrics/monitoring.

Local Cache Options

1. ConcurrentHashMap

Implementing a cache with JDK's ConcurrentHashMap is simple but lacks built‑in eviction, size limits, and expiration; additional custom code is required.

2. Guava Cache

Guava provides a mature cache library with maximum size, expiration after write or access, statistics, and LRU eviction.

com.google.guava
guava
31.1-jre

@Slf4j
public class GuavaCacheTest {
    public static void main(String[] args) throws ExecutionException {
        Cache
cache = CacheBuilder.newBuilder()
                .initialCapacity(5) // initial capacity
                .maximumSize(10)   // max entries, evict beyond
                .expireAfterWrite(60, TimeUnit.SECONDS) // expiration
                .build();

        String orderId = String.valueOf(123456789);
        // get orderInfo, compute if absent
        String orderInfo = cache.get(orderId, () -> getInfo(orderId));
        log.info("orderInfo = {}", orderInfo);
    }

    private static String getInfo(String orderId) {
        String info = "";
        // first query redis cache
        log.info("get data from redis");
        // if redis miss, query DB
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

3. Caffeine

Caffeine is a Java‑8‑based cache library that improves on Guava with a hybrid W‑TinyLFU eviction algorithm, offering near‑optimal performance.

com.github.ben-manes.caffeine
caffeine
2.9.3

@Slf4j
public class CaffeineTest {
    public static void main(String[] args) {
        Cache
cache = Caffeine.newBuilder()
                .initialCapacity(5)
                // evict when max size exceeded
                .maximumSize(10)
                // expire after write
                .expireAfterWrite(60, TimeUnit.SECONDS)
                .build();

        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId, key -> getInfo(key));
        System.out.println(orderInfo);
    }

    private static String getInfo(String orderId) {
        String info = "";
        // first query redis cache
        log.info("get data from redis");
        // if redis miss, query DB
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

4. Ehcache (Encache)

Ehcache is a pure‑Java in‑process cache with richer features such as multiple storage tiers (heap, off‑heap, disk), various eviction algorithms, and clustering support.

org.ehcache
ehcache
3.9.7

@Slf4j
public class EhcacheTest {
    private static final String ORDER_CACHE = "orderCache";

    public static void main(String[] args) {
        CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder()
                .withCache(ORDER_CACHE, CacheConfigurationBuilder
                        .newCacheConfigurationBuilder(String.class, String.class,
                                ResourcePoolsBuilder.heap(20)))
                .build(true);
        Cache
cache = cacheManager.getCache(ORDER_CACHE, String.class, String.class);

        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId);
        if (StrUtil.isBlank(orderInfo)) {
            orderInfo = getInfo(orderId);
            cache.put(orderId, orderInfo);
        }
        log.info("orderInfo = {}", orderInfo);
    }

    private static String getInfo(String orderId) {
        String info = "";
        // first query redis cache
        log.info("get data from redis");
        // if redis miss, query DB
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

Local Cache Issues and Solutions

1. Cache Consistency

Two‑level cache must stay consistent with the database; updates should invalidate both local and remote caches. Solutions include broadcasting invalidation messages via MQ or using Canal to capture binlog changes and propagate them.

2. Improving Hit Rate

Techniques for increasing local cache hit rate are discussed (reference omitted).

3. Technology Selection

From ease‑of‑use perspective, Guava, Caffeine, and Ehcache are all mature.

Functionally, Guava and Caffeine are similar (heap only), while Ehcache offers more features.

Performance tests show Caffeine is fastest, followed by Guava, with Ehcache lagging.

Overall, Caffeine is recommended as the primary local cache, combined with Redis or Memcached as the second‑level distributed cache.