How a Unified Precision Testing Platform Boosted Coverage and Efficiency Across Multiple Business Units

1. Introduction

The precision testing field faced many practical bottlenecks after more than a year of exploration across various business units (BUs). Problems included low coverage on legacy code, overwhelming change‑point data, and duplicated effort in building custom tools. This article introduces the internally open‑source Tianji platform and its concrete impact on the Yanxuan business line.

2. Tianji Platform Overview

The primary goal was to design a reusable, low‑cost solution that could be quickly adopted across the group. Tianji was co‑created by the Yanxuan and Media quality‑technology teams in 2021, launched for almost a year, and now serves over 300 applications, providing more than 40 000 coverage calculations and analyses.

2.1 Platform Characteristics

Unified data synchronization and visualization for all precision‑testing dependencies.

Hybrid public‑plus‑private deployment model to satisfy BU‑specific customization, compatibility, and security requirements.

2.2 Architecture

Public Platform : Offers a single entry point for all testing capabilities, handles scheduling, result storage, configuration management, and role‑based access control.

Agent (Private Deployment) : Executes testing functions in each BU’s environment, respecting code‑base, deployment, and version‑control constraints while providing a standard interface for customization.

SDK (Open‑Source) : Abstracts core testing operations, re‑implements underlying Jacoco and java‑callgraph algorithms, and exposes generic capabilities for integration into other toolchains.

2.3 Core Capabilities

Server‑side coverage collection, calculation, and reporting (Java, Golang).

Incremental coverage computation for branches and versions.

Incremental uncovered‑code analysis and result output.

Coverage comparison to highlight uncovered differences.

Cross‑project code‑diff analysis.

Automated test‑case association and filtering for changed code.

Impact analysis of changed interfaces, including upstream/downstream call chains.

TOP‑30 interface coverage monitoring.

Client‑side coverage (under development).

3. Yanxuan Deployment Practices

To address the identified pain points, Yanxuan applied two complementary strategies:

Subtraction : Refine the existing full‑coverage dataset to focus on the most critical and frequently exercised code paths, eliminating wasteful effort.

Addition : Enrich basic change‑analysis with additional context (interface importance, call‑graph, usage metrics) to guide developers and testers toward more accurate decisions.

3.1 Coverage Metrics

Incremental Coverage compares the current deployed version with the master branch, reporting two dimensions:

Changed‑line coverage : Coverage statistics only for lines marked as changed (+).

Changed‑method coverage : If a method contains any changed lines, the entire method’s line coverage is reported, while unchanged methods are excluded.

Effective Coverage leverages live traffic to identify which code paths are exercised in production, using the data to:

Guide automation to target high‑impact areas.

Provide developers with concrete evidence for deprecating redundant code.

TOP Interface Coverage tracks daily call volume for the most frequently invoked APIs and verifies, during release gating, whether automated tests cover these critical endpoints.

Release Gate Criteria (all thresholds must be met to proceed):

GoApi test‑case pass rate = 100 %

Overall line coverage ≥ 50 %

Incremental line coverage ≥ 70 %

Effective line coverage ≥ 60 %

TOP‑30 interface coverage ≥ 90 %

3.2 Code‑Change Analysis

The platform performs a diff between two versions, then uses byte‑code static analysis to identify the top‑level Controller or RPC entry points affected by each changed method. This enables rapid narrowing of the impact scope.

Upstream/downstream impact is further enriched by APM data, revealing call‑frequency and dependency graphs for the affected interfaces.

Interface importance is evaluated by combining online invocation counts with the number of impacted methods, assisting developers in prioritizing regression effort.

For each impacted interface, the platform links existing GoApi test cases (currently at URL granularity) and highlights gaps where high‑traffic interfaces lack test coverage, prompting targeted test creation.

4. Overall Benefits

After more than a year of continuous improvement, Yanxuan’s precision‑testing capabilities have evolved from version 1.0 (basic functions) to 2.0 (data‑driven decision making). Reported outcomes include:

Developer empowerment : Faster impact analysis for code changes, automated call‑graph extraction for large codebases (>1.49 M lines), and reduced manual effort in migration projects.

Tester empowerment : Quantifiable metrics for admission and release gates, 20‑30 % test efficiency gains, and integration of “effective coverage” into performance assessments.

Coverage statistics : Average line coverage across 130+ core services exceeds 55 %; effective coverage averages over 70 %; TOP‑30 interface coverage surpasses 90 %.

5. Future Roadmap

5.1 Intelligent Automation

Plans include combining traffic data from the Cloud Music referral platform, Hangzhou Research GoApi, and DevOps pipelines to create a “Precision Testing 3.0” that automatically generates test cases from real traffic, executes them in CI, and produces actionable reports for human review.

5.2 Open‑Source Expansion

The underlying atomic capabilities have already been abstracted for broad industry use and are co‑maintained by Yanxuan, Media, and Zhiji teams. The next step is to widen the open‑source scope beyond precision testing, inviting external contributors to co‑innovate and add value to the development lifecycle.