Design and Selection of Local In-Memory Cache Solutions for High-Performance Services

Background In high‑performance service architectures, caching is essential. Hot data is typically stored in remote caches such as Redis or Memcached, with database queries only on cache miss. To further improve response speed and reduce database pressure, a two‑level cache architecture combines a fast local cache (first‑level) with a remote cache (second‑level).

Why Use Local Cache

Local cache resides in process memory, offering extremely low latency for data that changes infrequently or does not require real‑time freshness.

It reduces network I/O by minimizing interactions with remote caches like Redis.

Essential Functions of a Local Cache

Store data with read/write capabilities.

Atomic (thread‑safe) operations, e.g., using ConcurrentHashMap .

Configurable maximum size.

Eviction policies such as LRU or LFU when the size limit is exceeded.

Expiration strategies (time‑based, lazy, scheduled).

Persistence support.

Statistics and monitoring.

Local Cache Implementation Options

1. Using ConcurrentHashMap

The cache is essentially a KV store in memory; ConcurrentHashMap provides thread safety. Additional logic is required for eviction, size limits, and expiration. This approach is simple and has no external dependencies, but custom development can be costly and may lack robustness for complex scenarios.

2. Guava Cache

Guava, an open‑source library from Google, offers a mature cache implementation with features such as maximum size, expiration based on write or access time, simple statistics, and LRU eviction.

Dependency:

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>31.1-jre</version>
</dependency>

Example usage:

@Slf4j
public class GuavaCacheTest {
    public static void main(String[] args) throws ExecutionException {
        Cache
cache = CacheBuilder.newBuilder()
                .initialCapacity(5) // initial capacity
                .maximumSize(10)     // max size, eviction after limit
                .expireAfterWrite(60, TimeUnit.SECONDS) // expiration
                .build();
        String orderId = String.valueOf(123456789);
        // fetch from cache or compute via getInfo()
        String orderInfo = cache.get(orderId, () -> getInfo(orderId));
        log.info("orderInfo = {}", orderInfo);
    }
    private static String getInfo(String orderId) {
        String info = "";
        // first query Redis
        log.info("get data from redis");
        // if not in Redis, query MySQL
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

3. Caffeine

Caffeine is a Java‑8 based cache library that improves on Guava with a hybrid W‑TinyLFU eviction algorithm, offering near‑optimal performance.

Dependency:

<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
    <version>2.9.3</version>
</dependency>

Example usage:

@Slf4j
public class CaffeineTest {
    public static void main(String[] args) {
        Cache
cache = Caffeine.newBuilder()
                .initialCapacity(5)
                .maximumSize(10) // eviction when exceeding size
                .expireAfterWrite(60, TimeUnit.SECONDS) // write expiration
                .build();
        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId, key -> getInfo(key));
        System.out.println(orderInfo);
    }
    private static String getInfo(String orderId) {
        String info = "";
        log.info("get data from redis");
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

4. Ehcache (referred to as "Encache" in the original text)

Ehcache is a pure‑Java in‑process cache framework, originally the default CacheProvider for Hibernate. It offers richer features than Guava or Caffeine, including multiple eviction algorithms (LRU, LFU, FIFO), heap/off‑heap/disk storage, persistence, and clustering support.

Dependency:

<dependency>
    <groupId>org.ehcache</groupId>
    <artifactId>ehcache</artifactId>
    <version>3.9.7</version>
</dependency>

Example usage:

@Slf4j
public class EhcacheTest {
    private static final String ORDER_CACHE = "orderCache";
    public static void main(String[] args) {
        CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder()
                .withCache(ORDER_CACHE, CacheConfigurationBuilder
                        .newCacheConfigurationBuilder(String.class, String.class,
                                ResourcePoolsBuilder.heap(20)))
                .build(true);
        Cache
cache = cacheManager.getCache(ORDER_CACHE, String.class, String.class);
        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId);
        if (StrUtil.isBlank(orderInfo)) {
            orderInfo = getInfo(orderId);
            cache.put(orderId, orderInfo);
        }
        log.info("orderInfo = {}", orderInfo);
    }
    private static String getInfo(String orderId) {
        String info = "";
        log.info("get data from redis");
        log.info("get data from mysql");
        info = String.format("{orderId=%s}", orderId);
        return info;
    }
}

Local Cache Issues and Solutions

1. Cache Consistency

Two‑level caches must stay consistent with the database. When data changes, both the local and remote caches should be updated.

Solution 1: MQ Broadcast – In a clustered deployment, publish a message to a message queue on data modification; each node consumes the message and invalidates its local cache, achieving eventual consistency.

Solution 2: Canal + MQ – Subscribe to MySQL binlog changes via Canal, then forward change events to MQ, allowing cache invalidation without embedding MQ logic in business code.

2. Improving Local Cache Hit Rate

Refer to general cache‑hit‑rate‑improvement techniques (e.g., proper key design, appropriate expiration policies).

3. Technical Selection of Local Cache Implementations

Ease of Use : Guava Cache, Caffeine, and Ehcache all provide mature integration guides and are easy to adopt.

Feature Set : Guava and Caffeine support only heap memory; Ehcache offers richer features such as off‑heap and disk storage.

Performance : Benchmarks show Caffeine has the best performance, followed by Guava, with Ehcache lagging behind.

Overall, Caffeine is recommended for most scenarios due to its superior speed, while Redis or Memcached should be used as the distributed second‑level cache to ensure reliability and scalability.