Alibaba Cloud DataWorks Intelligent Data Modeling: Practices, Challenges, and Solutions

DataWorks, Alibaba Cloud's big data development governance platform, has evolved for 14 years and recently launched the Intelligent Data Modeling tool, which leverages contributions from internal data warehouse teams such as Cainiao, Taobao, and Tmall.

1. Alibaba Data Demand Flow

The data warehouse construction involves three key roles: data demand owners (operations, BI, product managers), data product managers who translate business needs into data requirements, and data development engineers responsible for designing models and metrics.

2. Best Practices for Data Warehouse Modeling

Based on Kimball dimensional modeling, Alibaba adds a "business classification" layer to separate models by business team, defines "data domains" by aggregating business processes, and distinguishes between data marts (business, product, and public). Standards for table naming and field definitions are embedded in the product to ensure compliance.

DataWorks also provides a built‑in retail industry model covering common dimensions (order, member, product) and metrics, which can be imported directly into the modeling interface.

3. Hands‑On Modeling Demonstration

The demonstration shows the four steps of data modeling: warehouse planning, data standards, metric design, and dimensional modeling. It explains how to import source tables, perform data cleaning, and denormalize frequently used dimensions into DWD tables, as well as how to generate derived metrics in bulk.

For model modifications, code‑mode supports MaxCompute DDL, Hive DDL, and generating models from SELECT statements, enabling a seamless bridge between modeling and data development.

4. Common Modeling Challenges and Solutions

Cold‑start difficulty: conduct a comprehensive offline inventory of historical models, retire unused or duplicate models, and batch import cleaned models into DataWorks.

Standard enforcement: configure a modeling checker that forces table creation through the modeling tool, and automate table name generation based on naming rules.

Efficiency: batch generate derived metrics from atomic metrics and reuse them in summary and application layer models.

Design‑development gap: publish models as physical tables directly from the UI, automatically generate ETL code, and expose stable models in the Data Asset catalog for consumption.

5. Data Asset Application

After models are materialized, they appear in the Data Asset 3D panorama, allowing users to browse, select fields, and run zero‑code SQL analysis, with assets organized by domain for easy discovery.

6. Q&A

Q: Does DataWorks support slowly changing dimensions (SCD)? A: Automatic SCD generation is not yet publicly available. Q: How is data asset sharing handled? A: Administrators publish assets in the Data Asset module; ordinary users share assets via direct product links.

The Intelligent Data Modeling feature is now commercialized on Alibaba Cloud, with a personal version priced at 60 CNY for six months, including a retail e‑commerce template and tutorials. More details are at https://www.aliyun.com/product/bigdata/ide .