Design and Implementation of Qunar's Algorithm Service Platform for Machine Learning

In recent years, rapid advances in machine learning have increased its adoption in the internet industry, exposing challenges such as development standards, business boundary definition, feature data management, model training, and deployment. Qunar's large‑accommodation division created the Algorithm Service Platform to address these issues and improve development efficiency.

The platform, started in August 2016, aims to provide a visual management interface, standardized development guidelines, and unified workflow for machine‑learning projects, allowing algorithm engineers to focus on feature engineering and model training while business engineers can integrate services via simple APIs.

Since its inception, the platform has undergone three major iterations: the first version solved large‑scale model deployment, the second added feature pipeline management and sharing, and the current version emphasizes automated training, model deployment, and rapid iteration.

Design-wise, the platform consists of two main parts: the Algorithm Service Platform and the Model Training Platform. The workflow includes feature engineering, model training (often using Jupyter notebooks), automatic submission to the training platform, storage of model and feature metadata in a database, and exposure of services via RPC or client APIs.

Key functional modules include:

Model File Management: Automated pipeline from training to DB registration, validation, upload to a Swift‑based file system, and cleanup of obsolete versions.

Model Computation: A unified Evaluator interface supports various model types (Java‑reimplemented for compatibility) and allows custom extensions.

Feature Engineering: Separate handling of feature transformation (extending Airbnb's Aerosolve with Java) and feature collection pipelines for both offline batch and online low‑latency scenarios.

Algorithm Service: Two deployment modes—proxy and RPC—provide flexibility in performance, resource usage, and monitoring.

Monitoring: Comprehensive metrics for service performance, feature quality, and data collection using InfluxDB, Chronograf, and Kapacitor, with alerts via QT and email.

The automated training platform enables engineers to submit notebook code to a Git repository, configure jobs through a unified UI, and trigger data‑driven training pipelines, eliminating manual scheduling and supporting nested model training.

As of the latest snapshot, the platform hosts 60 online applications, 4,522 model files (807 independent models), 170 GB of public features, and 260 GB of algorithm‑specific features, delivering roughly 450 QPS of algorithm services for the division.