Key Considerations and Implementation Strategies for a Local Cache in Java

Preface

While studying MyBatis source code, I noticed its caching subsystem, which offers both first‑level and second‑level caches. The second‑level cache is more feature‑rich and serves as a good reference for building a local cache, though it may differ slightly from dedicated cache frameworks such as Ehcache.

Considerations

The main aspects to think about when designing a local cache are the storage data structure, maximum capacity, handling of excess entries, expiration strategy, thread safety, blocking behavior, a simple public API, and whether persistence is required.

1. Data Structure

The simplest approach is to store entries in a Map . More sophisticated systems (e.g., Redis) provide additional structures such as hashes, lists, sets, and sorted sets, which are built on top of linked lists, zip lists, hash tables, and skip lists.

2. Object Limit

Because a local cache lives in memory, it usually defines a maximum number of cached objects (e.g., 1024). When this limit is reached, a removal strategy must be applied.

3. Eviction Policy

Common eviction strategies include LRU (Least Recently Used), FIFO (First In First Out), LFU (Least Frequently Used), SOFT (soft references), and WEAK (weak references).

4. Expiration Time

In addition to eviction, caches often support a per‑entry TTL. Expired entries can be removed lazily during get/put operations or proactively by a background job.

5. Thread Safety

Unlike Redis, which runs single‑threaded, a local cache may be accessed concurrently, so thread‑safe data structures (e.g., ConcurrentHashMap ) or explicit synchronization (e.g., MyBatis' SynchronizedCache ) are required.

6. Simple API

A user‑friendly cache should expose a minimal set of operations such as get , put , remove , clear , and getSize . MyBatis defines a Cache interface, while Guava provides a richer Cache API.

7. Persistence

Persisting cache data to disk allows recovery after a restart. Ehcache supports disk persistence, and Redis offers AOF and RDB persistence mechanisms.

8. Blocking Mechanism

When a cache miss occurs, it can be useful to block other threads from performing the same expensive computation. MyBatis provides a BlockingCache implementation, and a classic memoizer pattern using ConcurrentHashMap with FutureTask can achieve the same effect.

How to Implement

Below are concrete implementation snippets for each consideration.

1. Data Structure

Most local caches use a Map for storage. MyBatis' second‑level cache uses a HashMap wrapped by SynchronizedCache for thread safety:

Map<Object, Object> cache = new ConcurrentHashMap<Object, Object>();

2. Object Limit

A default maximum size (e.g., 1024) is applied when the user does not specify a limit. Once the limit is reached, the chosen eviction policy is triggered.

3. Eviction Policy

Typical policies are implemented as follows:

LRU – often realized with LinkedHashMap . FIFO – implemented with a Queue . LFU – requires a HashMap that records access frequencies. SOFT – uses SoftReference . WEAK – uses WeakReference .

4. Expiration Time

Two approaches are common:

Passive removal checks the TTL on each get / put :

if (System.currentTimeMillis() - lastClear > clearInterval) {
    clear();
}

Active removal runs a background job that periodically scans and deletes expired entries.

5. Thread Safety

Use thread‑safe collections or synchronize critical sections. MyBatis' SynchronizedCache example:

public synchronized void putObject(Object key, Object object) { ... }
public synchronized Object getObject(Object key) { ... }

6. Simple API

MyBatis defines a minimal Cache interface:

public interface Cache {
    String getId();
    void putObject(Object key, Object value);
    Object getObject(Object key);
    Object removeObject(Object key);
    void clear();
    int getSize();
    ReadWriteLock getReadWriteLock();
}

Guava's Cache API provides additional convenience methods such as getIfPresent , invalidate , and statistics.

7. Persistence

Persisting to disk can be toggled via configuration, e.g., diskPersistent="false" for Ehcache. Redis offers AOF and RDB persistence.

8. Blocking Mechanism

A memoizer implementation that ensures a single computation per key:

public class Memoizer<A, V> implements Computable<A, V> {
    private final Map
> cache = new ConcurrentHashMap<>();
    private final Computable
c;
    public Memoizer(Computable
c) { this.c = c; }
    @Override
    public V compute(A arg) throws InterruptedException, ExecutionException {
        while (true) {
            Future
f = cache.get(arg);
            if (f == null) {
                Callable
eval = () -> c.compute(arg);
                FutureTask
ft = new FutureTask<>(eval);
                f = cache.putIfAbsent(arg, ft);
                if (f == null) { f = ft; ft.run(); }
            }
            try { return f.get(); }
            catch (CancellationException e) { cache.remove(arg, f); }
        }
    }
}

Conclusion

The article has presented the major design points for a local cache: data structure, size limit, eviction policy, expiration handling, thread safety, blocking behavior, a concise API, and optional persistence. Additional considerations may exist, and readers are encouraged to contribute further ideas.