Datav: Building an Efficient Data Visualization Platform for Dashboard Development

Shopee’s rapid data growth has made traditional spreadsheet analysis insufficient, creating a strong need for automated dashboard generation. Manual front‑end development of dashboard pages consumes excessive human and time resources, often causing delays in responding to business requirements.

This article describes the design and implementation of Datav , an internal data‑visualization platform that aims to halve project cycles (from ~40 days to ~20 days) and dramatically reduce front‑end effort (from 10 person‑days to 3 person‑days per dashboard).

Key objectives :

Shorten project cycles.

Reduce manpower, especially back‑end and QA involvement.

Provide a low‑code environment for designers, product managers, and front‑end engineers.

Overall architecture consists of five subsystems:

Designer : a low‑code editor that captures layout, interaction, and component data configuration using a DSL.

Admin : platform management (data computation, asset management, permission control, publishing).

UI Components : a library of >50 reusable chart components built on a standard DSL protocol.

Datav Server : a Node.js service handling authentication, data aggregation, and dynamic SQL generation.

Datasource Access Server : connectors for various data sources (MySQL, ClickHouse, future support for Kafka, Elasticsearch, etc.).

The platform enables direct data‑source connections, allowing components to query raw tables without a back‑end API. Example SQL generation for a two‑dimension chart is shown below:

Select [indicator] from [table_xxx] group by [dimension1], [dimension2]

For a single‑dimension chart:

Select [indicator] from [table_xxx] group by [dimension1]

More complex queries, including pagination and ordering, are also generated automatically:

SELECT field1 FROM Demo-Table GROUP BY date_day LIMIT 1000 OFFSET 100 ORDER BY field3 ASC

Datav also supports metadata computation, allowing users to define derived fields, aliases, and custom SQL‑based data themes. The data‑management module includes source management, field editing, and theme configuration, with visual and SQL‑based options.

Component interaction is handled through visual linkage or optional JavaScript code. The visual approach establishes relationships between filter and display components, listens for parameter changes, and triggers data refreshes, reducing the need for manual coding.

Benefits :

Project duration reduced from ~40 days to ~20 days.

Front‑end effort cut by ~200 % (10 person‑days → 3 person‑days).

Total development effort lowered from 33 person‑days/day to 13 person‑days/day (≈150 % efficiency gain).

Standardized component library, Node middleware, CI scripts, and finer‑grained permission systems.

The article concludes that Datav significantly improves dashboard development efficiency, shortens cycles, and provides a scalable foundation for future data‑driven products.