How a Tiny HashMap Bug Triggered a Massive Memory Leak in a High‑Traffic Microservice

A new architect with extensive high‑concurrency experience joined the team and was tasked with the "highest concurrency" requirement: collecting the average response time and total request count for every API via an AOP interceptor.

The implementation stored monitoring data in a

ConcurrentHashMap

without any comments, assuming the simple code needed no further documentation.

After deployment, the service began to suffer from memory overflow. Using Eclipse MAT and

jmap

, the team discovered millions of

MonitorKey

and

MonitorValue

objects lingering in the heap.

<code>Monitor$MonitorKey@15aeb7ab</code>

The root cause was that

MonitorKey

did not override

equals

and

hashCode

, so identical keys were treated as distinct, causing unbounded growth of map entries.

In addition, the

visit

method performed a non‑atomic sequence: retrieve the key, check for null, create a new value, and put it back. Concurrent threads could interleave, leading to lost updates:

<code>Thread1: get value for key a → null → create b → put a=b
Thread2: get value for key a → null → create c → put a=c</code>

Two remediation approaches were discussed:

Adding

synchronized

to the method:

<code>public synchronized void visit(String url, String desc, long timeCost) { ... }</code>

Using

putIfAbsent

with atomic counters:

<code>MonitorKey key = new MonitorKey(url, desc);
MonitorValue value = monitors.putIfAbsent(key, new MonitorValue());
value.count.getAndIncrement();
value.totalTime.getAndAdd(timeCost);
value.avgTime = value.totalTime.get() / value.count.get();</code>

The technical director favored the synchronized version for its simplicity, while the team argued for the more efficient

putIfAbsent

approach.

After fixing the

equals

/

hashCode

implementation and applying a proper atomic update strategy, the memory leak disappeared and the service stabilized.

Key lessons: always override

hashCode

and

equals

for objects used as map keys, ensure map operations are atomic in high‑concurrency scenarios, and prefer minimal, safe changes when fixing production bugs.