Kuaishou’s Data Application Factory: Boosting BI with Low‑Code & Unified Queries

Background Introduction

Kuaishou’s data platform shared its best practices in the "2022 DataFun Kuaishou Metrics Middle Platform" forum, focusing on the challenges of fast‑growing business demands, diverse data applications, and the need for rapid delivery, data accuracy, timeliness, and service stability.

Overall Design

The system is built on the metrics middle platform and consists of three main parts:

Low‑code : DSL‑based rapid delivery for specialized analysis products.

Middle‑platform capabilities : Unified query adapters, caching, and secondary calculations encapsulated as reports.

Multi‑engine support : Extensible integration of various query engines.

Key Technologies

Unified Query

To address data source diversity and fragmented metrics, a unified query service was built on the metrics middle platform, supporting ClickHouse, Druid, APIs, and providing caching and secondary computation layers. Queries are routed synchronously or asynchronously, with cache checks for synchronous queries and background engine queries for asynchronous paths.

Query Optimization

Query aggregation based on data‑source granularity reduces engine I/O and network overhead. Examples include merging multiple metric requests to the same source into a single query. A hierarchical cache (real‑time and offline) further speeds up repeated queries, while cache pre‑heating based on historical usage improves first‑time query performance.

Low‑Code DSL

A JSON‑based DSL describes interactive charts, layout, component linkage, event listeners, and fine‑grained configurations. This enables front‑end rendering without extensive post‑processing, standardizing chart protocols and reducing development effort.

Summary and Planning

The factory has built a one‑stop, minimal‑configuration metrics management and application platform, achieving over 400 data sets, 3,000 metrics, and 2,000 charts, with a 35% component customization rate. Quality, efficiency, and capability have improved, supporting core business lines such as the main site, e‑commerce, commercialization, local life, and overseas.

Future directions include personalized low‑code customization (function compute, advanced components, ecosystem building) and graphical orchestration (rich components, drag‑and‑drop, intelligent parameters, WYSIWYG).

Q&A

Q: How does the Data Application Factory differ from KwaiBI? A: KwaiBI serves generic scenarios, while the factory handles complex, business‑specific analytics that require higher customization and interaction.

Q: Does caching provide significant benefits over direct database queries? A: Yes; most queries target pre‑aggregated metrics, and caching combined with MPP engines like ClickHouse greatly improves latency.

Q: What does the DSL look like? A: It is a JSON structure that fully describes an interactive chart, as detailed in the low‑code section.

Q: What is delivered by the factory? A: Complete data‑application products including filters, charts, and interactive metrics.