Analysis and Optimization of G1GC Long Pauses Caused by MySQL Driver Phantom References

The application experienced occasional GC pauses of up to 4 hours, with request latency exceeding 3 seconds, traced to improper G1GC settings combined with MySQL connections that linger beyond their idle timeout, generating a large number of phantom references that prolong the G1 mixed‑GC marking phase.

Environment : JDK 1.8, mysql‑connector‑java‑5.1.30, commons‑dbcp‑1.4, Spring 4.3.20, 8‑core CPU, 16 GB RAM.

JVM startup parameters (original) :

-Xms9984m -Xmx9984m -XX:MaxMetaspaceSize=512m -XX:MetaspaceSize=512m -XX:MaxDirectMemorySize=512m
-XX:ParallelGCThreads=8 -XX:CICompilerCount=4 -XX:ConcGCThreads=4 -server -XX:+UnlockExperimentalVMOptions
-XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:SurvivorRatio=10 -XX:MaxTenuringThreshold=5
-XX:InitiatingHeapOccupancyPercent=45 -XX:G1ReservePercent=20 -XX:G1MixedGCLiveThresholdPercent=80
-XX:MaxGCPauseMillis=100 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+PrintReferenceGC -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -Xloggc:/export/Logs/app/tomcat7-gc.log

DBCP configuration (excerpt):

maxActive=20
initialSize=10
maxIdle=10
minIdle=5
minEvictableIdleTimeMillis=180000
timeBetweenEvictionRunsMillis=20000
validationQuery=select 1

The GC logs show a young GC of 0.10 s followed by a remark phase lasting 3.77 s, mainly due to processing a huge number of PhantomReference objects originating from com.mysql.jdbc.NonRegisteringDriver which stores connections in a static ConcurrentHashMap<ConnectionPhantomReference, ConnectionPhantomReference> .

MySQL driver source highlights :

public Connection connect(String url, Properties info) throws SQLException { … }
NonRegisteringDriver.trackConnection(this);
protected static final ConcurrentHashMap<ConnectionPhantomReference, ConnectionPhantomReference> connectionPhantomRefs = new ConcurrentHashMap();
protected static void trackConnection(com.mysql.jdbc.Connection newConn) { ConnectionPhantomReference phantomRef = new ConnectionPhantomReference((ConnectionImpl)newConn, refQueue); connectionPhantomRefs.put(phantomRef, phantomRef); }
static { AbandonedConnectionCleanupThread referenceThread = new AbandonedConnectionCleanupThread(); referenceThread.setDaemon(true); referenceThread.start(); }

Because idle connections are reclaimed after only 3 minutes ( minEvictableIdleTimeMillis=180000 ), many connections become phantom‑referenced and accumulate in the map, causing the long STW pauses during GC.

Remediation :

Enable parallel reference processing: -XX:+ParallelRefProcEnabled (JDK 8 required).

Increase ParallelGCThreads to match CPU cores (8) or use -XX:+UseDynamicNumberOfGCThreads .

Lower G1MixedGCLiveThresholdPercent from 80 % to 60 % to trigger more frequent but shorter mixed GCs.

Adjust MaxTenuringThreshold to 15 (or default) to avoid premature promotion.

Increase minEvictableIdleTimeMillis to 30 minutes to reduce phantom reference creation.

Option A: Periodically clear connectionPhantomRefs via reflection (e.g., every 2 hours).

Option B: Upgrade MySQL Connector/J to 8.0.22+ and disable the abandoned‑connection cleanup thread with -Dcom.mysql.cj.disableAbandonedConnectionCleanup=true .

After applying the JVM tweaks and addressing the MySQL driver phantom‑reference issue, GC pause times dropped to an average of 26 ms, staying well below the 100 ms target, and mixed‑GC frequency was dramatically reduced.

References include several online articles on G1GC tuning, MySQL driver memory leaks, and JVM performance handbooks.