How Pre‑Signing Parameters and a Thread‑Safe Queue Slash API Test Latency

Problem

During API load testing a thread often spends a noticeable amount of time between two consecutive requests. The extra latency comes from operations such as result verification, logging, or data aggregation, but the dominant cost is the cryptographic signing of request parameters. In the cited tests the signing step took around 10 ms and could reach 100 ms under high concurrency, creating a CPU bottleneck.

Solution Overview

The latency is eliminated by pre‑computing the signature for every request before the request is sent. The signed parameter maps are stored in a thread‑safe LinkedBlockingDeque. Producer threads generate and enqueue the ready‑to‑send maps, while consumer threads dequeue them and perform the HTTP POST. This trades a modest amount of memory for a large reduction in per‑request latency.

Implementation Details

The following Java program demonstrates the complete approach.

package com.okayqa.others.payyst;

import com.alibaba.fastjson.JSON;
import com.fun.base.constaint.ThreadLimitTimesCount;
import com.fun.frame.excute.Concurrent;
import com.fun.frame.httpclient.FanLibrary;
import com.fun.utils.ArgsUtil;
import com.fun.utils.RString;
import com.okayqa.common.RSAUtilLJT;
import com.okayqa.common.Users;
import org.apache.http.client.methods.HttpPost;
import org.slf4j.Logger;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.atomic.AtomicInteger;

public class T extends FanLibrary {
    private static final Logger logger = getLogger(T.class);
    /** Queue that holds pre‑signed request parameters */
    private static final LinkedBlockingDeque<Map<String, String>> paramQueue = new LinkedBlockingDeque<>();
    /** Counter used to generate unique order numbers */
    private static final AtomicInteger counter = new AtomicInteger(111000);

    public static void main(String[] args) throws Exception {
        ArgsUtil argsUtil = new ArgsUtil(args);
        int threadCount = argsUtil.getIntOrdefault(0, 1);
        int times = argsUtil.getIntOrdefault(1, 100);

        // ---------- producers: pre‑sign parameters ----------
        List<Runnable> producers = new ArrayList<>();
        for (int i = 0; i < threadCount; i++) {
            producers.add(new Producer(times));
        }
        new Concurrent(producers, "Parameter pre‑initialization").start();

        // ---------- consumers: send API requests ----------
        List<Runnable> consumers = new ArrayList<>();
        for (int i = 0; i < threadCount; i++) {
            consumers.add(new Consumer(times));
        }
        new Concurrent(consumers, "Member payment & renewal API").start();

        testOver();
    }

    /** Build a map of request parameters and sign it with RSA. */
    public static Map<String, String> getParams() {
        int idx = counter.getAndIncrement();
        Map<String, String> p = new HashMap<>();
        p.put("days", "1");
        p.put("memberId", "208");
        p.put("orderNo", "F" + RString.getString(4) + idx);
        p.put("orderPaySystemId", "85123213");
        p.put("orderPayTime", "2020-02-09 10:00:00");
        p.put("payMoney", "30");
        p.put("recordSources", "3");
        p.put("renewal", "false");
        String user = Users.getStuUser(idx % 1000);
        p.put("systemId", user);
        // RSA signature – the expensive operation that we pre‑compute.
        String sign = RSAUtilLJT.sign(p, RSAUtilLJT.getPrivateKey(RSAUtilLJT.RSA_PRIVATE_KEY));
        p.put("sign", sign);
        return p;
    }

    /** Producer thread: generate signed maps and put them into the queue. */
    private static class Producer extends ThreadLimitTimesCount {
        Producer(int times) { super(null, times, null); }
        @Override
        protected void doing() throws Exception {
            Map<String, String> signed = getParams();
            paramQueue.add(signed);
        }
    }

    /** Consumer thread: take a signed map and execute the HTTP POST. */
    private static class Consumer extends ThreadLimitTimesCount {
        private static final Logger logger = getLogger(Consumer.class);
        Consumer(int times) { super(null, times, null); }
        @Override
        protected void doing() throws Exception {
            String url = com.okayqa.studentapd.base.OkayBase.HOST + "/api/member/createOrRenewMember";
            // take() blocks until a pre‑signed map is available
            Map<String, String> params = paramQueue.take();
            HttpPost post = getHttpPost(url, JSON.toJSONString(params));
            String requestId = "F" + getNanoMark();
            post.addHeader(getHeader("requestid", requestId));
            String response = FanLibrary.excuteSimlple(post);
            if (!response.contains("success")) {
                logger.warn(requestId + " response not successful: " + response);
                fail();
            }
        }
    }
}

Key points:

Pre‑signing : The RSA signature is calculated once per request in the producer stage, removing the costly operation from the critical request‑sending path.

Thread‑safe queue : LinkedBlockingDeque guarantees FIFO ordering and blocks consumers when the queue is empty, preventing duplicate work and ensuring safe concurrent access.

Atomic counter : Guarantees unique orderNo values across all threads.

Separation of concerns : Producers focus on CPU‑intensive signing, while consumers handle I/O‑bound HTTP calls, improving overall throughput under high concurrency.

The method was validated in a previous micro‑benchmark article that measured the impact of signing on load‑test results, confirming that pre‑signing and queuing produce stable, low‑latency API calls.