Taobao Big Data Model Governance and DataWorks Co‑development

The Taobao technical data system has grown significantly, supporting complex business scenarios, but rapid data expansion has exposed problems such as non‑standard table naming, low reuse of common‑layer tables, and inefficient application‑layer data usage.

Key issues identified include: lack of model governance on the product side, rising storage costs, reduced efficiency, weakened standards, and increased operational burden.

Four aspects of data health were analyzed – evaluation, construction, management, and usage – revealing gaps in unified assessment, missing end‑to‑end modeling tools, and insufficient cost and reuse control.

To address these challenges, a set of goals was defined: digitize model evaluation, push common‑layer data down, productize a full‑cycle modeling platform, and improve data retrieval efficiency.

Through close co‑creation with the DataWorks team, a comprehensive solution was built, including:

DataWorks intelligent data modeling (warehouse planning, dimension modeling, forward/reverse modeling, multi‑engine publishing).

Standardized table naming and governance policies.

Model scoring and dashboard for health monitoring.

Enhanced search & recommendation, data albums, and table description upgrades.

Development assistant for permission reminders, release control, and temporary table automation.

Key achievements after FY22 include a model evaluation system, intelligent modeling capabilities, upgraded data map functions, and defined governance processes.

Future plans focus on refining the Taobao technical model architecture, further improving intelligent modeling, expanding data map integration, enhancing the development assistant, and automating model/table decommissioning.