Full-Field Interface Automation Validation for Dubbo Services

Background: Interface automation is a critical part of quality assurance. With rapid business growth, the team faces challenges in achieving timely, complete, and maintainable validation of returned fields in automated test cases.

Pain points include the large number of fields in product models, duplicated validation logic, evolving importance of previously non‑core fields (e.g., product code), and the risk of default values overwriting updates.

Goal: Perform exhaustive validation of all fields returned by test cases while significantly improving test‑case authoring efficiency.

Scenario analysis reveals two types of backend APIs: operation APIs that write data and query APIs that retrieve large sets of business fields. The solution treats these two categories separately.

Preparation: The article outlines the current testing environment at Youzan, which uses a weak isolation strategy with a base environment and a project (SC) environment. Requests are routed based on a flag (sc) to determine which environment should handle the request.

Practice

Read‑interface validation: Send the same request to both base and project environments and compare the responses. If they match, the test passes. The approach leverages AOP to intercept Dubbo calls before and after execution.

Write‑interface validation: After a write operation, the resulting data can be retrieved via one or more read interfaces. The write is executed in both environments; the subsequent reads are compared after ignoring fields that differ due to randomness.

Read‑interface validation flow

The flow diagram (omitted) shows that the key step is generating a list of fields to ignore by comparing two base‑environment responses.

Object‑to‑Map conversion code (shown below) extracts a map of JSON‑path‑like keys to values, enabling map‑based comparison.

public class ItemSavedModel {
    private Long itemId;
    private Long shopId;
}

Core utility for building the path map:

/**
 * Get object path map
 */
public static HashMap
getObjectPathMap(Object obj) {
    HashMap
map = new HashMap<>();
    getPathMap(obj, "", map);
    return map;
}

private static void getPathMap(Object obj, String path, HashMap
pathMap) {
    if (obj == null) {
        return;
    }
    Class
clazz = obj.getClass();
    // primitive
    if (clazz.isPrimitive()) {
        pathMap.put(path, obj);
        return;
    }
    // wrapper types
    if (ReflectUtil.isBasicType(clazz)) {
        pathMap.put(path, obj);
        return;
    }
    // collection or map
    if (ReflectUtil.isCollectionOrMap(clazz)) {
        if (Map.class.isAssignableFrom(clazz)) {
            Map map = (Map) obj;
            map.forEach((k, v) -> {
                if (k != null) {
                    getPathMap(v, path + "/" + k.toString(), pathMap);
                }
            });
        } else {
            Object[] array = ReflectUtil.convertToArray(obj);
            for (int i = 0; i < array.length; i++) {
                getPathMap(array[i], path + "/" + i, pathMap);
            }
        }
        return;
    }
    // POJO fields
    List
fields = ReflectUtil.getAllFields(clazz);
    fields.forEach(field -> getPathMap(ReflectUtil.getField(obj, field), path + "/" + field.getName(), pathMap));
    return;
}

Write‑interface validation: Executes three rounds of requests. The first two rounds run in the base environment to compute ignored fields and capture baseline responses. The third round runs in the project environment; after removing ignored fields, the responses are compared to determine pass/fail.

Key implementation details include:

Controlling retry timing to avoid non‑idempotent writes before data cleanup.

Triggering the second execution round via a custom TestNG suite listener (code shown).

Using an annotation (@WriteCaseDiffAnnotation) and an IMethodInterceptor to ensure only write‑type test cases run in the first two suites.

// Trigger three requests
public class GlobalCoverISuiteListener implements ISuiteListener {

    public static ConcurrentHashMap
suiteFinishMap = new ConcurrentHashMap<>();

    @Override
    public void onStart(ISuite suite) {
        if(suiteFinishMap.size()==0 ){
            System.setProperty("globalCoverFlag", "1");
            if(System.getProperty("globalCoverFlag").equals("1")) {
                suiteFinishMap.put(suite.getXmlSuite().getName(),1);
                TestNG tng = new TestNG();
                tng.setXmlSuites(Arrays.asList(suite.getXmlSuite()));
                tng.run();
            }
        }
    }

    @Override
    public void onFinish(ISuite suite) {
        suite.getResults().forEach((suitename, suiteResult)->{
            ITestContext context = suiteResult.getTestContext();
            if(System.getProperty("globalCoverFlag").equals("1")) {
                int before = suiteFinishMap.get(suite.getXmlSuite().getName());
                if(suiteFinishMap.get(suite.getXmlSuite().getName())==2){
                    suiteFinishMap.put(suite.getXmlSuite().getName(),++before);
                    System.setProperty("globalCoverFlag", "0");
                    return;
                }
                suiteFinishMap.put(suite.getXmlSuite().getName(),++before);
                TestNG tng = new TestNG();
                tng.setXmlSuites(Arrays.asList(context.getCurrentXmlTest().getSuite()));
                tng.run();
            }
        });
    }
}

Insufficiencies: Currently only supports Dubbo interfaces; future work may extend to front‑end Node APIs. It also heavily relies on the base testing environment, which could be decoupled by introducing a storage‑based approach.

Acknowledgments: Thanks to the product testing team members for technical and design support. Recruitment information is provided.

Further reading links are listed for related topics such as mock service plugins, Dubbo pressure testing, and CI containerization practices.