Impala Optimization and Practices at NetEase Big Data Platform

NetEase selected Impala as the OLAP engine for its big‑data platform because of its MPP decentralized architecture, excellent query performance, user‑friendly Web UI, full Hive metadata compatibility, active Apache community, support for multiple data formats, and integration with Kudu for real‑time warehousing.

The team enhanced Impala by adding a management server to persist WebUI information in MySQL, implementing automatic metadata synchronization that consumes Hive DDL logs to trigger INVALIDATE METADATA, and introducing a Zookeeper‑based routing layer for high‑availability across coordinators.

Additional optimizations include support for Iceberg tables, Alluxio as a secondary cache, and Elasticsearch integration, as well as various operational improvements such as separating coordinator and executor nodes, configuring queues per business or SQL type, and using hints to influence join strategies.

In practice, Impala is deployed in NetEase's "Mammoth" data platform and Cloud Music service, with clusters ranging from mixed to independent HDFS deployments, often combined with Kudu for real‑time data. The system serves self‑service analytics, BI reporting, A/B testing, and commercialized data services, achieving sub‑second query latency for most workloads.