Troubleshooting DDL Hang Issues in Dble: Step-by-Step Analysis
This article outlines a systematic approach to diagnosing and resolving DDL execution hangs in Dble, covering log inspection, context analysis, identifying problematic data nodes and MySQL connections, and confirming lock contention as the root cause.
The article begins by posing the problem: when a DDL statement executed through Dble hangs, how can we determine whether it is caused by user actions or a Dble bug.
Step 1: Check Dble logs for errors or warnings. An alert in the log matches the observed hang, indicating a problem.
Step 2: Examine the surrounding log context to understand Dble's processing stages. By filtering the log (e.g., less dble.log|grep DDL ), we see that the DDL is sent to four shards and involves two phases: connection testing and actual DDL execution.
The logs show that the connection test succeeded, but one node remains in the start state. The problematic connection is identified as connection 23, which maps to data node dn2 .
Further inspection of the MySQL instance on that node (using show @@processlist ) reveals that the DDL is waiting for a lock to be released.
Conclusion: The DDL hang is caused by lock contention on one of the MySQL nodes, not by a Dble bug.
Summary of troubleshooting steps:
Inspect Dble logs for relevant errors or alerts.
Analyze the log context to understand which processing stage the issue occurs in.
Locate the affected data node and MySQL connection based on log details.
On the identified MySQL node, run show @@processlist to confirm lock waiting.
Aikesheng Open Source Community
The Aikesheng Open Source Community provides stable, enterprise‑grade MySQL open‑source tools and services, releases a premium open‑source component each year (1024), and continuously operates and maintains them.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.