NL2SQL from a Database Perspective: Overview, History, and Laboratory Projects (GAR, MetaSQL, PURPLE)

Introduction NL2SQL (Natural Language to SQL) converts natural language queries into executable SQL statements, enabling users to retrieve data without writing code. Traditional query workflows rely on experts, leading to inefficiency and limited accessibility.

Why NL2SQL? It addresses challenges such as complex SQL syntax, non‑intuitive schemas, numerous operators, diverse user needs, and high learning costs for DBMS expertise.

Application Scenarios NL2SQL can enhance BI tools, power customer‑service chatbots, and support data‑driven decision making for non‑technical users.

Key Technical Components Effective NL2SQL systems require schema linking, SQL generation, and query fixing/optimization.

Evaluation Metrics Common metrics include Exact‑set Match (EM), Execution Match (EX), and Valid Efficiency Score (VES) to assess correctness and performance.

Development History The field evolved from rule‑based systems (1980s‑1990s) to model‑based approaches (2010s) and, more recently, large‑language‑model (LLM) based methods (2020s), with increasing focus on task decomposition and multi‑agent collaboration.

Laboratory Projects

GAR – Generates candidate SQLs, converts them to natural language (SQL2NL), ranks candidates, and executes the top result.

MetaSQL – Enhances autoregressive decoding with metadata guidance and a two‑stage ranking pipeline.

PURPLE – An end‑to‑end LLM pipeline that prunes schemas, predicts SQL skeletons, selects exemplars for prompting, and applies heuristic post‑processing to ensure correctness.

Experimental Findings GAR achieves strong results; MetaSQL improves performance on benchmarks like Spider; PURPLE matches or exceeds fine‑tuned models on EM, EX, and TS metrics while remaining robust across different model sizes and budgets.

Future Outlook Planned directions include agents with NL2SQL capabilities, specialized NL2SQL agents, automatic SQL correction, explainability, broader database adaptation, handling complex BI queries, and improving robustness.

Q&A The session addresses differences between NL2SQL and Text‑to‑SQL, performance benchmarks, handling complex BI SQL, data generation pipelines, and recommended datasets such as Spider, BIRD, and their variants.