Design and Selection of Local In‑Memory Caches for High‑Performance Service Architecture

Background

In high‑performance service architecture, caching is indispensable. Hot data is usually stored in remote caches like Redis or Memcached, and only when a cache miss occurs is the database queried, which speeds up access and reduces database pressure.

Why Use Local Cache

Local cache resides in the process memory, offering extremely fast access for data with low change frequency and low real‑time requirements.

It reduces network I/O by avoiding unnecessary interactions with remote caches such as Redis.

Required Functions of a Local In‑Memory Cache

Store data with read/write capability.

Provide atomic (thread‑safe) operations, e.g., using ConcurrentHashMap .

Support a maximum size limit.

Offer eviction strategies when the limit is exceeded, such as LRU or LFU.

Support expiration policies (time‑based, lazy, periodic).

Optional persistence.

Statistics and monitoring.

Local Cache Solution Options

1. Using ConcurrentHashMap as Local Cache

The essence of a cache is a KV structure stored in memory, which can be implemented with Java's thread‑safe ConcurrentHashMap . However, additional features like eviction, size limits, and expiration must be custom‑developed, making it suitable only for simple scenarios.

2. Guava Cache

Guava, an open‑source library from Google, provides a robust cache implementation with features such as maximum size limits, two expiration policies (based on insertion time or access time), simple statistics, and LRU eviction.

<dependency>
   <groupId>com.google.guava</groupId>
   <artifactId>guava</artifactId>
   <version>31.1-jre</version>
</dependency>

@Slf4j
public class GuavaCacheTest {
    public static void main(String[] args) throws ExecutionException {
        Cache<String, String> cache = CacheBuilder.newBuilder()
                .initialCapacity(5)  // initial capacity
                .maximumSize(10)     // max entries, evict when exceeded
                .expireAfterWrite(60, TimeUnit.SECONDS) // expiration
                .build();
        String orderId = String.valueOf(123456789);
        // Get orderInfo; if key missing, callable fetches data
        String orderInfo = cache.get(orderId, () -> getInfo(orderId));
        log.info("orderInfo = {}", orderInfo);
    }
    private static String getInfo(String orderId) {
        String info = "";
        // first query Redis cache
        log.info("get data from redis");
        // if Redis miss, query DB
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

3. Caffeine

Caffeine is a Java‑8‑based next‑generation cache library whose performance is close to the theoretical optimum. It can be seen as an enhanced Guava Cache, using a hybrid LRU/LFU algorithm (W‑TinyLFU) for superior performance.

<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
    <version>2.9.3</version>
</dependency>

@Slf4j
public class CaffeineTest {
    public static void main(String[] args) {
        Cache<String, String> cache = Caffeine.newBuilder()
                .initialCapacity(5)
                .maximumSize(10)               // evict when exceeded
                .expireAfterWrite(60, TimeUnit.SECONDS) // write expiration
                //.expireAfterAccess(17, TimeUnit.SECONDS) // rarely used
                .build();
        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId, key -> getInfo(key));
        System.out.println(orderInfo);
    }
    private static String getInfo(String orderId) {
        String info = "";
        log.info("get data from redis");
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

4. Ehcache

Ehcache is a pure‑Java in‑process cache framework that offers fast, lightweight caching and is the default CacheProvider for Hibernate. Compared with Guava and Caffeine, it provides richer features such as multiple storage tiers (heap, off‑heap, disk) and clustering support.

<dependency>
   <groupId>org.ehcache</groupId>
   <artifactId>ehcache</artifactId>
   <version>3.9.7</version>
</dependency>

@Slf4j
public class EhcacheTest {
    private static final String ORDER_CACHE = "orderCache";
    public static void main(String[] args) {
        CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder()
                .withCache(ORDER_CACHE, CacheConfigurationBuilder
                        .newCacheConfigurationBuilder(String.class, String.class, ResourcePoolsBuilder.heap(20)))
                .build(true);
        Cache<String, String> cache = cacheManager.getCache(ORDER_CACHE, String.class, String.class);
        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId);
        if (StrUtil.isBlank(orderInfo)) {
            orderInfo = getInfo(orderId);
            cache.put(orderId, orderInfo);
        }
        log.info("orderInfo = {}", orderInfo);
    }
    private static String getInfo(String orderId) {
        String info = "";
        log.info("get data from redis");
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

Local Cache Issues and Solutions

1. Cache Consistency

Two‑level caches and the database must stay consistent; when data changes, both the local and remote caches should be updated synchronously.

Solution 1: MQ

In a clustered deployment, each node has its own local cache. Using a broadcast‑style MQ, a data‑change event is published, and all nodes consume the message to invalidate their local caches, achieving eventual consistency.

Solution 2: Canal + MQ

If you prefer not to embed MQ logic in business code, you can subscribe to database change logs. Canal captures MySQL binlog events, publishes them to MQ, and the cache layer consumes the messages to keep caches consistent.

2. Improving Local Cache Hit Rate

Refer to best‑practice guides on how to increase cache hit ratios.

3. Technical Selection of Local Memory

From an ease‑of‑use perspective, Guava Cache, Caffeine, and Ehcache all provide mature integration and are simple to adopt.

Functionally, Guava Cache and Caffeine are similar (heap‑only), while Ehcache offers richer features such as off‑heap and disk storage.

Performance-wise, Caffeine is the fastest, followed by Guava Cache; Ehcache shows the lowest throughput in benchmark results.

For local caching, Caffeine is highly recommended due to its superior performance. Although Ehcache provides richer capabilities such as persistence and clustering, these can be achieved by other means. In real‑world projects, combine Caffeine for local caching with Redis or Memcached for distributed caching to build a robust multi‑level cache architecture.