Intelligent Test Localization Practices: Spectrum-Based Fault Localization, Error-Code Build System, Revenue‑Loss Decision, and UI Case Localization

Previous articles introduced the five steps of testing activities (test input, execution, analysis, localization, and evaluation); this chapter focuses on the intelligent practice of the test localization step.

Test localization aims to quickly identify the cause of a failure after it occurs, helping responders choose appropriate mitigation actions, reducing handling time and labor costs. It is generally divided into two types: (1) root‑cause localization, which pinpoints the actual reason such as a code defect, and (2) behavior localization, which identifies the operation or change that led to the failure for rapid remediation.

Intelligent test localization combines data, algorithms, and engineering techniques to analyze problem symptoms, related data, and system information, using strategies or algorithms to suggest the triggering operation or root cause, thereby guiding decision‑making for issue resolution. Baidu QA has been exploring and applying these methods.

1. Spectrum‑Based Fault Localization – This approach uses instrumented builds to execute test suites, collecting the four‑tuple <ef, ep, nf, np> for each instrumented statement (where ef = executions in failing tests, ep = executions in passing tests, nf = non‑executions in failing tests, np = non‑executions in passing tests). Suspiciousness scores are computed with formulas such as Tarantula, Ochiai, and Overlap, and statements are ranked to produce a high‑risk code fragment list. The technique is generic and applicable to unit, functional, and diff testing, significantly lowering manual debugging effort, and is being integrated into Baidu’s automated testing pipeline.

2. Error‑Code Based Build‑System Localization – In large‑scale automation, many abnormal builds require manual labeling and fixing. By treating error codes as concise representations of issues, the system can automatically annotate error categories, trigger self‑healing actions (e.g., automatic restarts), and create issue tickets for human closure. This involves three strategies: automatic annotation (mapping error codes to categories), self‑healing (defining trigger conditions and sub‑strategies), and issue‑closure (auto‑creating cards with error details for manual verification). Pilot projects achieved a 100% abnormal task reporting rate and a 94% closure rate.

3. Commercial Revenue‑Loss Decision Localization – This workflow consists of alarm reception, diagnosis, fault‑feature extraction, loss decision, and recommendation. It emphasizes coverage (alarm and metric coverage) and diagnostic strategies (risk level identification, risk‑metric mapping, abnormal point detection, log‑trace localization, PV loss estimation, and loss‑mitigation plan generation). The system allows businesses to customize diagnostic logic, providing end‑to‑end alarm triggering, diagnosis, and loss‑mitigation recommendations.

4. Search UI Display Case‑Level Localization – To improve system quality, comprehensive monitoring is established, but case‑level localization can be resource‑intensive. The solution builds a complete log‑trace system with seed‑only storage, module topology reconstruction, and regex‑based extraction, achieving second‑level latency and minute‑level alarm reception with automated pinpointing of problematic UI cases.

The article concludes with links to related “Technical Fuel Station” series articles covering intelligent testing in evaluation, analysis, automation execution, automatic generation, and large‑scale deployment.