Tencent Cloud Native Database AI Autonomy: SIGMOD Research and Intelligent Tuning System

Tencent Cloud's native database team research has been accepted to SIGMOD, marking a major breakthrough in database autonomy. This article introduces the integration of AI technology into databases to form an autonomous database brain.

Professor Zhou Ke from Huazhong University of Science and Technology explains that with massive data growth, DBA (human operations) cannot keep pace with data growth. Database workloads are diverse and dynamic, requiring adaptive management. The basic framework for database autonomy includes three aspects: observation, analysis, and decision-making. The goal is to enable autonomous databases to automatically configure, manage, and optimize for specific data and workloads without human assistance.

Future challenges for database autonomy include: 1) Efficient adaptability for dynamic and diverse workloads; 2) Performance stability with fewer resources; 3) Interpretability of autonomous operations to help system administrators learn and optimize.

CDBTune (Tencent Cloud MySQL Hybrid Tuning System) uses deep reinforcement learning to automatically tune database parameters end-to-end. Unlike existing methods, it does not require workload type classification or large sample accumulation. The system uses genetic algorithms and expert experience for fast preheating, with parallel architecture significantly improving tuning speed.

The engineering implementation separates service scheduling from task execution. The Learner task extracts samples, computes network gradients, updates neural networks, and recommends database parameters to Actors. Actors interact with training instances, set parameters, replay traffic, and collect performance data, forming a closed loop.

Tencent Cloud MySQL's AI-powered capabilities include: automatic anomaly detection (memory analysis, kernel instrumentation, IO latency hardware resource statistics), SQL throttling for abnormal requests, Compressed histogram for accurate statistics, Statement Outline for固化 query plans, parallel index building optimization (up to 15x speedup), optimizer autonomy with automatic optimization strategy generation, deadlock scenario optimization, flash sale scenario optimization (50x performance improvement), and HA switching optimization with master-slave cache synchronization.

Tencent Cloud MySQL published two SIGMOD papers: in 2019, CDBTune was accepted as a Research Full Paper; in 2022, the HUNTER paper was accepted, demonstrating further breakthroughs in database AI intelligence.