Automated Test Data Construction Using Metersphere, APIs, and Kafka

During testing, many business scenarios require massive amounts of test data, such as statistical and billing data over a month, to verify trend changes and accuracy. Data can be generated via database SQL, business APIs, Kafka consumption, or scripting tools.

When a system exposes data APIs, the preferred method is to call those interfaces, which avoids dirty data and ensures realism. Using API testing tools, timers, and data‑construction scripts, one can simulate user activity without manual entry.

Scenario 1 – Real‑time Data Construction

For IoT devices that report online status, activation, and user binding, data originates from database tables, Redis, and batch jobs. By configuring Metersphere timers to invoke the business API at different times of day, hourly active‑device counts are generated automatically, creating a natural trend.

Steps include importing device information from a CSV file, selecting a subset of devices via a controller, passing CSV variables as parameters, creating a timer with second‑level precision, and adding a batch‑statistics API call that runs hourly.

Scenario 2 – Billing Data Generation via Kafka

To simulate a month’s worth of order data in Kafka, a custom Java JAR provides an HTTP endpoint that writes JSON payloads to a Kafka topic. Metersphere’s loop controller, combined with environment variables for start/end dates, generates random timestamps, order numbers, and other fields.

The process involves configuring the loop controller, adding pre‑script code to produce random dates and unique order IDs, constructing the order JSON, and invoking the Kafka‑write API to push the data.

Summary

Various methods exist for test data construction: direct platform operations, online data import, API‑driven automation, database scripts, and hybrid approaches. Choosing the right technique depends on the business context, desired data realism, and effort constraints, with API calls and automated tools offering the most efficient and accurate results.