When AI Becomes Its Own Data Engineer: Inside DataMaster

Recent AI research has shifted from a human‑driven pipeline—collecting data, cleaning it, writing training code, and designing experiments—to a scenario where AI participates directly in the R&D loop, writing code, fixing bugs, invoking tools, and iteratively improving based on feedback.

This trend first appeared in code generation and experimental automation, and now extends to data engineering. The paper "DataMaster: Data‑Centric Autonomous AI Research" (GitHub: https://github.com/sjtu-sai-agents/DataMaster, arXiv: https://arxiv.org/abs/2605.10906) proposes the role of an AI Data Engineer that operates when models and training algorithms are fixed.

DataMaster lets an agent around a given task automatically discover external data sources, filter them, clean and transform the data, and construct training inputs, continuously iterating based on downstream model feedback while keeping the model and training algorithm unchanged.

The system is built around three core components:

Data Tree : explores multiple data‑engineering paths. Red nodes scout new data sources; black nodes clean, transform, and combine data into trainable versions.

Data Pool : stores all discovered data sources so that any branch can reuse them later.

Global Memory : records each attempt—data used, processing steps, training scores, failure reasons—and enables the agent to avoid starting from scratch.

These components turn data engineering from a one‑off script into a searchable, revisitable, and continuously optimizable process, analogous to a data‑team workflow.

Experimental validation focuses on whether autonomous data iteration alone can improve downstream performance.

In the MLE‑Bench Lite scenario—where the task provides fixed data and training code—the baseline medal rate is 35.91%. DataMaster raises it to 68.18% (+32.27 points) and the gold‑medal rate from 22.73% to 45.45%.

In the PostTrainBench scenario—representing post‑training of large models with no ready data—DataMaster lifts the average score from 8.47% to 31.17% (+22.70 points), achieving the highest average among baselines.

For the GPQA sub‑task, which tests graduate‑level scientific reasoning, DataMaster improves the score from 18.75% to 31.0%. The improvement grows with increased search budget, as the agent discovers and integrates more relevant scientific, reasoning, and MedQA data. This result surpasses the expert‑instruction model reference (30.35%) and baselines such as Codex, DataFlex, and ML‑Master 2.0.

To rule out data leakage, the authors filtered benchmark‑related sources, performed hash‑based deduplication, and found no exact or fuzzy matches among the 7,479 training samples, with n‑gram overlap remaining between 0.08% and 1.06%.

The authors argue that data engineering is not a pre‑training prerequisite but a core decision loop: the system decides what data to acquire, how to process it, how to combine it, and continuously refines the data strategy based on downstream feedback.

When data becomes an AI decision object, new challenges arise—data provenance, compliance, test‑set contamination, and auditability—requiring transparent and controllable data‑management processes.

Ultimately, DataMaster demonstrates that autonomous data engineering can substantially boost performance without altering models or training algorithms, highlighting the emerging importance of letting AI manage its own data.