Designing an Intelligent Performance Testing Platform: From Vision to Implementation

Introduction

During the bank’s IT architecture transformation—from mainframes to open platforms and from centralized to distributed systems—the need for higher quality assurance arose due to increased performance stability challenges and complex service calls. Traditional testing relied heavily on manual steps, limiting breadth, depth, and efficiency.

Although mainstream tools like JMeter, LoadRunner, Tsung, and nGrinder were tried, each exhibited shortcomings such as poor large‑scale support, steep learning curves, limited protocol coverage, high cost, and fragmented functionality.

Design Principles of an Intelligent Performance Platform

The ideal platform should provide at least the following capabilities:

Test demand identification : automatic risk detection and test requirement generation.

Test case design : automated generation, templating, version control, and integration.

Test data construction : automatic creation, cleaning, aggregation, and asset management.

Multi‑protocol & complex scenario support : support for HTTP, TCP, JDBC, JMS, etc., with free composition.

Large‑scale concurrency simulation : emulate massive simultaneous users.

Intelligent execution & monitoring : automated test execution, real‑time resource monitoring, data collection, and archiving.

Chaos state simulation : inject chaos scenarios during high concurrency to test robustness.

Result analysis & visualization : automated analysis, reporting, and visual dashboards.

Test management & monitoring : manage tasks, resources, progress, and reports.

Intelligent capacity planning : predict future performance needs from trends.

Technical Implementation

The platform was built on JMeter as the execution engine, extended with a master‑slave architecture, Prometheus monitoring, and custom diagnostic models.

1. Demand/Risk Identification

Using a decision‑tree algorithm, the system evaluates factors such as core program changes (via ASM bytecode + DFS), database capacity trends, and operational alerts, then calculates a Gini‑based risk level to select services for testing.

2. Smart Test Case Design

Test cases are auto‑generated from performance indicators, historical and production service profiles, monitoring scenarios, and expert rule libraries, linking each service to multiple test scenarios (O=container, F=metric, M=model, Z=business metrics).

3. Intelligent Data Construction

Data templates combined with database metadata and machine‑learning clustering produce synthetic test data, while automated workflows manage metadata, templates, and service‑transaction mappings.

4. Multi‑Scenario/Protocol Support

By rewriting JMeter component parsing logic, about 90% of components can be edited and assembled online, enabling complex transaction scenarios through a web interface.

5. High Concurrency Support

The master‑slave mode splits test tasks across multiple containers, allowing the platform to exceed the capacity of a single JMeter instance.

6. Intelligent Execution

Business metrics and system resources are automatically linked, collected, and displayed, with real‑time monitoring that snapshots the system when performance bottlenecks are detected.

7. Chaos Injection

Integrating the ChaosBlade framework provides delay and packet‑loss injection for Dubbo, SQL, container networks, CPU, and memory, enabling staged chaos testing during performance runs.

8. Smart Result Analysis

Historical models, production data, and expert rules evaluate test outcomes, generate alerts, and produce flame‑graphs for code‑level bottleneck analysis; reports are auto‑generated from templates.

9. Asset Management & Monitoring

The platform tracks test cases, data, and results for traceability, supports periodic model retraining, and manages test resources via auto‑scaling PaaS containers.

Overall, the platform now covers the ten capability areas described earlier, providing a comprehensive, intelligent solution for performance testing.