Quick Diagnosis of MySQL Hang Issues Using Callstack and Mutex Analysis
This article demonstrates a step‑by‑step method for diagnosing MySQL hangs by collecting callstack information, identifying mutex conflicts, and mapping function calls to code locations to pinpoint the root cause of deadlocks.
Do you often encounter MySQL hangs and feel helpless? This article shows, through a concrete case, how to quickly analyze such problems.
When MySQL hangs, it is usually caused by internal mutex conflicts. Before restarting the service, you should collect the callstack information.
pstack `pidof mysqld` > mysql_callstackNote: the mysqld binary must contain symbol tables.
Analysis Steps
1. Filter the callstack log to find, for each thread, the function called just before inline_mysql_mutex_lock and the corresponding mutex code location, which indicates the mutex the thread is waiting for.
2. Traverse backward from that function through each caller to see which mutexes have already been successfully acquired.
3. Finally, trace each lock‑acquiring function back to the front‑end operation that triggered it and create a table showing which threads hold and wait for which mutexes, making the conflict relationships clear.
Conclusion
The hang was caused by concurrent operations—show binary logs, purge binary logs, and reading performance_schema.session_variables —that each acquire different mutexes, leading to a deadlock where new connections cannot be created.
show binary logs: holds LOCK_log , waits for LOCK_index
binlog purge: holds LOCK_index , waits for LOCK_thd_data
reading performance_schema.session_variables : holds LOCK_thd_data and LOCK_global_system_variables , waits for LOCK_log
new connection: waits for LOCK_global_system_variables
The root cause was that reading binlog_transaction_dependency_* variables required LOCK_log , which easily caused deadlocks; this issue was fixed in MySQL 5.7.25.
Click the original link to view the analysis script.
Aikesheng Open Source Community
The Aikesheng Open Source Community provides stable, enterprise‑grade MySQL open‑source tools and services, releases a premium open‑source component each year (1024), and continuously operates and maintains them.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.