Design and Evolution of Zhuanzhuan's R&D Measurement Platform

1. What the Measurement Platform Does

The platform systematically collects key R&D data, visualizes delivery value, efficiency, and quality, provides a reliable observation system, discovers potential issues, and supports business improvements.

2. Technical Construction History

2.1 Platform V0.1 (2019‑2022)

Collected demand, bug, project delay data to assist efficiency analysis; simple maintenance without continuous updates.

2.2 Platform V0.5 (2022‑2023)

Re‑architected to address slow queries, low metric accuracy, and production latency. Key changes included decoupling data model from business logic, redesigning tables into detail, dimension, bridge, and summary tables, and implementing data governance processes.

2.2.1 Reconstruction Idea

Separated data model and business logic, stored complete transaction details, enabling flexible data management and better traceability.

2.2.2 Basic Data Governance

Standardized project process data from TAPD, promoted unified fields, and integrated process standards into the development workflow to ensure consistent data collection.

2.2.3 Achieved Effects

Pre‑computed data solved query‑slow problems.

Metric accuracy improved, though overall effect still limited due to remaining calculation logic in code.

2.3 Platform V1 (2023‑Present)

Shifted to DW/BI‑driven data modules with three core capabilities: unified data source, SQL‑as‑metric, and efficient interactive design, aiming for accurate data, one‑hour data delivery, and easy analysis.

3. Platform V1 Details

3.1 Technical Architecture

Offline processing using Hive, StarRocks, Spark, and internal platforms (星河, 星火).

3.2 Unified Data Source (Data Warehouse Construction)

Mapped existing metrics to source fields, built a metric‑source matrix, and evolved from star schema to constellation model to support multiple fact tables.

3.2.2 Dimensional Modeling

Followed Kimball’s four‑step process to define business processes, granularity, and dimensions, creating fact and dimension tables for fine‑grained analysis.

3.2.3 Data Warehouse Layering

Implemented ODS, DW, DWS, and ADS layers to ensure data quality, aggregation, and fast OLAP querying.

3.2.4 Dimension Table Construction

Designed demand dimension tables with primary and auxiliary tables, handling slowly changing dimensions (SCD 1‑3) to preserve historical changes.

3.3 SQL‑as‑Metric

Metrics are defined directly by SQL statements, enabling transparent, reusable, and maintainable indicator calculation.

3.4 Efficient Interaction Design

Interactive dashboards with filtering, small‑card overviews, linked legends, drill‑down capabilities, and cross‑chart interactions to help users quickly locate anomalies and perform correlation analysis.

4. Summary

The platform improves data accuracy, rapid data delivery, and analysis by leveraging DW/BI systems, allowing the team to focus on metric management and UI interaction, supporting thousands of charts and thousands of derived metrics.

5. Future Plans

Simplify backend configuration and interaction design.

Reduce user onboarding cost through training and documentation.