NetEase Cloud Music Real-Time Data Warehouse Architecture and Low‑Code Platform Practices

The presentation begins with an introduction to NetEase Cloud Music's real‑time scenarios, describing four key aspects: real‑time features, monitoring, data products, and business decision layers, followed by a detailed dimension‑model‑based real‑time data warehouse architecture comprising ODS, CDM, ADS, and business decision layers.

It then explains the stream‑batch consistency approach, covering the Lambda architecture, challenges of maintaining model consistency across streaming and batch pipelines, and the design of a partition flow table that maps partition fields to Kafka topics with static and dynamic partition management.

The next section discusses the low‑code practice, comparing Kappa, DataPhin, and the adopted low‑code platform, and introduces the FastX architecture with its model, configuration, execution, operation, and service layers, emphasizing model‑driven development, component‑based pipelines, configuration simplifications, and service‑oriented lineage.

Finally, the article outlines future plans, including strengthening infrastructure, integrating low‑code with stream‑batch processing, exploring lake‑house solutions, and enhancing high‑availability through multi‑cluster setups and real‑time asset governance.