Designing Performance Test Scenarios: Models, Metrics, and Strategies

Performance testing is a complex system engineering activity that requires comprehensive knowledge to identify bottlenecks and propose optimizations. A complete test workflow includes requirement analysis, environment preparation, script development, data seeding, scenario design, execution, monitoring, bottleneck analysis, fixing, regression testing, and reporting.

Scenario Design is the core of performance testing. A scenario is equivalent to a functional test case and defines the actions to achieve specific test goals. Essential elements include the test model, metrics, transactions, scripts, concurrent users, pressure strategy, run time, delay method, user termination, and monitoring strategy (see Figure 1).

Test Model and Metrics are defined before scenario design. Metrics for web applications typically cover online users, optimal and maximum concurrent users, average response time, and target TPS. The test model represents the proportion of transaction volumes rather than concurrent users, reflecting real request pressure.

Transactions and Scripts must be prepared with realistic data (e.g., usernames, card numbers). Each script should contain a single complete transaction to simplify TPS calculation and maintain clear transaction ratios.

Concurrent Users (virtual users) are derived from the target TPS and transaction response time (TPS = concurrent users / response time). The required number of users is calculated for each transaction based on its proportion in the test model.

Pressure Strategies include simultaneous start, timed ramp‑up, and gradient loading. Simultaneous start stresses the system abruptly, while timed ramp‑up mimics gradual traffic increase. Gradient loading sets predefined user steps to discover the maximum achievable TPS.

Run Time varies by scenario type; typical durations are listed in Table 1 (e.g., several minutes for baseline, hours for stability).

Delay Methods control the interval between requests to achieve precise TPS. Three types are used: zero delay, fixed delay, and interval‑type delay (the latter ensures TPS stability even when response times vary).

User Termination mirrors the loading method, commonly using simultaneous exit to observe resource release behavior.

Monitoring must cover OS resources (CPU, memory, disk I/O, network), database metrics (TOP SQL, locks), and JVM statistics (GC, threads, connection pool). Sampling frequency is adjusted to scenario length.

Common Scenario Types include:

Single‑transaction baseline: one user, zero delay, long run to obtain reference response time.

Single‑transaction load: multiple users to find optimal TPS while keeping CPU usage ≤ 70%.

Mixed‑load: multiple transactions with proportions from the test model, aiming for optimal TPS.

Mixed‑capacity: extend mixed‑load to discover the maximum TPS the system can sustain.

Large‑concurrency: 5‑10× optimal concurrency to test system behavior under extreme load.

Stability: constant pressure for extended periods (hours) to detect memory leaks or performance drift.

Scalability: compare TPS on single‑node vs. multi‑node deployments (good scalability ≈ 1.8× TPS).

Reliability/exception: inject faults (e.g., network glitches) under load to verify recovery.

Impact testing: compare TPS with and without background asynchronous processing.

Shield‑delay comparison: use a proxy to add artificial latency and observe effects.

Online‑user scenario: maintain logged‑in idle users to test session handling.

Optimal and maximum concurrency scenarios to identify the user count at peak TPS and the absolute limit.

Gradient pressure: incrementally increase users until TPS plateaus, useful for quickly finding maximum throughput.

The article concludes that scenario design is an iterative process: results from one scenario feed into the next, adjusting concurrency, delay, and monitoring settings until the performance goals are met.