Baidu's Group-Level Continuous Delivery Pipeline: Dual-Engine Ecosystem and Intelligent Build

1. Baidu Build Development History

1.1 What is Build

Software build is a critical phase in the software lifecycle, spanning from source collection, compilation, testing to final product delivery, directly affecting quality and efficiency. It involves continuous gathering of source code, translation, testing, and delivery, with this article focusing on the technical aspects.

1.2 Baidu Build Development History

Before 2009 Baidu mainly used a waterfall model, which suffered from strong upstream/downstream dependencies, late delivery, merge conflicts, and manual testing. Transitioning to agile development introduced automated testing tools, CI, and Jenkins clusters per business line. With the rise of mobile products, Baidu built numerous mobile testing platforms and, from 2015, launched a mobile CI solution. Since 2017, Baidu established engineering capability standards, a dual-engine CI/CD ecosystem (containerized and plugin‑based), and began moving toward intelligent testing.

1.3 New Problems

After 2017 Baidu faced new challenges:

Business side: New product speed, technologies, and complex architectures demand new build capabilities. Higher iteration frequency requires improved build efficiency. Diverse business types and tech stacks have differing build requirements.

Technical side: Increased build volume reduces stability and success rate. Longer build chains increase manual effort for exceptions. Resource scarcity pressures build efficiency. Scattered tool platforms cause management and optimization difficulties. Version inconsistencies and chaotic management affect stability.

To address these, Baidu began exploring a one‑stop pipeline open platform with unified governance.

2. Baidu Dual‑Engine Ecosystem Solution

2.1 Engineering Capability Standards

Ba​idu’s DevOps‑TOC defined a set of engineering capability standards covering SERVER, SDK/APP, AI, and FRONTEND categories, spanning requirements, architecture design, development, testing, operations, and release. The dual‑engine ecosystem classifies and governs tools based on these standards.

2.2 Standard Build Engine – Plugin Center

The plugin center unifies delivery platforms, reducing the cost of maintaining separate CI platforms. It operates as an open platform where tools register services and become directly usable in pipelines. Governance, stability, and reliability standards are enforced by DevOps‑TOC.

2.2.1 Fast Access to Standard Plugins

Tools integrate via a standardized plugin interface (invoke, poll/callback, cancel). Business users configure global, plugin, and custom parameters to create dynamic execution scenarios, enabling low‑cost, rapid plugin consumption.

2.2.2 Obtaining Stable High‑Quality Plugins

Plugins undergo TOC review and certification, linking stability and capability to usage scope. Post‑integration, TOC monitors health metrics, providing evaluation data for continuous improvement.

2.3 Non‑Standard Build Engine – CI Containerization

Long‑tail tasks (short‑duration, low‑stability, low resource utilization) are handled by a K8s + Docker dynamic cluster solution. Key features include on‑demand container creation (2‑3 seconds), isolation for stability, reduced queuing, and efficient resource usage.

Speed: Fast container spin‑up.

Stability: Uniform images ensure consistent environments.

Queue reduction: Dynamic allocation and release of containers.

Optimizations: Container cache, disaster recovery, multi‑region resource pools, image pre‑distribution, high concurrency execution.

The combined plugin‑based and containerized engines handle roughly 80 % and 20 % of build traffic respectively, improving standardization, stability, resource utilization, and tool reuse, while generating structured data for future intelligent build enhancements.

3. Intelligent Build

3.1 Origin of Intelligent Build

Rapid business iteration exposed inefficiencies in traditional pipelines: fixed execution flows, redundant steps, wasted resources, and manual intervention for failures. Dynamic, policy‑driven execution is needed to boost efficiency.

3.2 On‑Demand Dynamic Execution

Ba​idu defines a “strategy” as an abstracted operation inserted into pipelines, capable of analyzing data and influencing task execution before, during, or after a build. Strategies may be analytical (non‑state‑changing) or decision‑making (state‑changing) and can target individual tasks, similar tasks, or entire pipelines. The platform resolves dependency chains, triggers execution, and processes results.

3.3 Long‑Tail Build Closed Loop

After a build, the platform offers a customizable issue‑closure workflow: automatic analysis, labeling, and self‑healing strategies reduce manual debugging and accelerate problem resolution.

3.4 Business Interaction Logic of Intelligent Build

The flow is: pipeline triggers the middle‑platform, which performs pre‑analysis to inform scheduling decisions; tasks execute based on strategy outcomes; upon completion, risk analysis runs and issues are classified and reported.

4. Summary

The article reviews Baidu’s build evolution, the dual‑engine ecosystem introduced after 2017, and the intelligent build platform that leverages data‑driven policies. The solution is now deployed across Baidu’s major business lines, supporting over 80 plugins, nearly 2 000 code repositories, hundreds of strategies, and tens of thousands of daily builds, delivering significant efficiency gains.