Analyzing QPS Calculation Errors in Fixed‑Thread Models

In previous posts I introduced two formulas for calculating QPS. This article investigates the error issues of these formulas under a fixed‑thread model.

QPS = total requests divided by total time, i.e.:
QPS = count(r)/T

QPS = number of threads divided by average response time, i.e.:
QPS = thread/rt

Calculation Model

The diagram shows a simple model of a single thread handling a single request, divided into three parts: before (pre‑request), request and response , and after (post‑request). Here T represents the total time of the three parts, while rt denotes the time spent in the request‑response phase.

Error Sources

Theoretical Error

This error originates from the before and after phases, which differ from the before() and after() methods in the FunTester framework. They represent various processing steps performed before each request and after each request.

Pre‑request time consumption includes data loading, parameter handling, and request setup such as assembling the URL, obtaining string and numeric parameters, and constructing the HttpRequestBase object. Although usually short, longer operations may involve reading MySQL, handling massive data sets, locked data, or complex parameter calculations.

Post‑request time consumption mainly involves parsing the response and performing assertions. Using regular expressions or script engines (e.g., an SDK ) for response parsing can be time‑consuming. See the article "JMeter Throughput Error Analysis" for examples.

In the article "Correcting Load Test Results with Micro‑benchmarks", a parameter signature caused excessive time consumption, leading to large test‑result errors that required baseline micro‑benchmark correction.

Actual Error

This error type is based on practical experience and includes factors that theoretically have little impact but cause noticeable deviations in real work. The idealized request model shown earlier is an example.

Below are some common sources:

Log Printing

Both normal logs and file I/O during testing contribute to this error. Large performance‑test data volumes generate massive logs; if all logs are written to files without filtering, the overhead can be significant. For example, JMeter logs can reach gigabytes, and serial log writing can noticeably affect test timing.

Many testers ignore JMeter system logs due to their volume and instead use APIs or custom components to output selective logs. I previously wrote Groovy scripts to handle JMeter assertions and logs, recording only abnormal requests. While small data volumes incur microsecond‑level overhead, large‑scale logging reintroduces the same error.

Real‑time Data Processing

Some test frameworks display real‑time metrics such as QPS, rt, 95th percentile, and total counts, as well as server‑side indicators. Calculating and rendering these metrics in the GUI consumes considerable hardware resources, often causing the test to stall. I therefore prefer post‑processing: collecting raw data during the test and performing statistical analysis after the test completes.

To be continued…

FunTester , Tencent Cloud Annual Author , Boss Zhipin Signed Author , GDevOps Official Media Partner , non‑famous test developer.

2021 Automation Testing Trends (Translation)

Appium 2.0 Overview

FunTester Framework Architecture Overview

FunTester Project Architecture Overview

Single‑Link Performance Testing Practice

Six‑Step Automation Strategy

Implementing QPS Sampler in Performance Test Framework

Exploring Branch Issues in Link Load Testing

Don’t Waste Your Curiosity

Correcting Load Test Results with Micro‑benchmarks

Click to read the original article and view the author's historical posts.