Didi's Presto Engine: Architecture, Optimizations, and Operational Practices

Didi has been developing its internal Presto engine for over three years, making it the preferred solution for ad‑hoc and Hive‑SQL acceleration, serving more than 6,000 users and processing 2–3 PB of HDFS data daily.

Presto Overview

Presto is an open‑source massive parallel processing (MPP) SQL engine originally from Facebook, designed for fast, real‑time analytics. It separates compute from storage and accesses various data sources via a Connector SPI.

Architecture

Presto follows a master‑slave model with a single Coordinator and multiple Workers. The Coordinator parses SQL, creates execution plans, and schedules tasks, while Workers execute the tasks. Connectors (e.g., Hive, MySQL, Kudu, Kafka) enable data source access, and custom connectors can be added.

Low‑Latency Principles

Presto achieves low latency through fully in‑memory parallel computation, pipelining, data locality, dynamic plan compilation, careful memory and data‑structure usage, GC control, and a design without fault tolerance.

Application at Didi

Presto is used for Hive‑SQL acceleration, ad‑hoc queries, BI and custom reports, marketing activities, data quality checks, asset management, and fixed‑data products. The platform now handles 30 trillion+ records per day.

Cluster Deployment

Didi operates mixed and high‑performance clusters. Mixed clusters share HDFS with offline Hadoop, using label‑based physical isolation to prevent large queries from affecting small ones. High‑performance clusters have dedicated HDFS and can query Druid for real‑time data.

Integration Methods

Multiple client interfaces (JDBC, Go, Python, CLI, R, Node.js, HTTP) have been developed, integrating with Didi's internal permission system for seamless access across tech stacks.

Engine Iteration

Since 2017, Didi has tracked 56 Presto releases, moving from PrestoDB to PrestoSQL (now Trino) for its active community, richer PR response, and better suitability for ad‑hoc workloads.

Hive SQL Compatibility

To ease migration from Hive, Didi added implicit type conversion, semantic and syntax compatibility, Hive view support, Parquet/HDFS reading, extensive UDFs, and achieved a 97‑99% SQL success rate with 30‑50% performance gains and up to 10× speedups in some cases.

Physical Resource Isolation

Didi introduced label‑based scheduling, allowing queries to be directed to specific worker groups, thus isolating high‑resource workloads and guaranteeing performance for latency‑sensitive services.

Druid Connector

Presto‑on‑Druid was implemented to overcome HDFS latency and low QPS, providing real‑time data access, leveraging Druid’s pre‑aggregation, caching, and push‑down capabilities (limit, filter, project, aggregation). This yielded 4‑5× performance improvements for relevant workloads.

Usability Enhancements

Features such as tenant‑based permission management via Ranger, support for add‑partition, numeric table/column names, view and alias authorization, and extended syntax (e.g., add partition) were added to improve developer experience.

Stability Construction

Monitoring was built using Presto plugins for audit logging, JMX metrics exported to Ganglia, logs stored in HDFS/Elasticsearch, and UI improvements for worker health. Common issues (Coordinator OOM, Worker GC, TCP TIME_WAIT, JVM bugs) were identified and mitigated with configuration tuning, JVM upgrades, and resource‑group controls.

Engine Optimization and Research

Performance gains were achieved through JVM parallel Ref‑Proc execution, ORC bloom filters, data grooming, partition pruning, and push‑down of limit, filter, project, and aggregation to storage. Experiments with Presto‑on‑Alluxio and Presto‑on‑CarbonData were abandoned due to memory overhead and stability concerns.

Future Outlook

Upcoming work includes addressing nighttime resource under‑utilization, upgrading to PrestoSQL 340, and open‑sourcing the Presto‑on‑Druid connector to contribute back to the community.

Conclusion

Didi's Presto has become the primary ad‑hoc query engine, delivering >10× latency reductions for key products and maintaining sub‑2 second average query times on high‑performance clusters.