Server‑Side Request Deduplication in Java Using Redis and MD5

Duplicate user requests, especially write operations, can cause serious issues such as double ordering; the article first describes typical duplication scenarios like replay attacks, client‑side resubmissions, and gateway retries.

When a request carries a unique identifier, the simplest solution is to store that identifier in Redis with a short TTL; if the key already exists, the request is considered a duplicate. The following code shows the basic Redis SETNX pattern:

String KEY = "REQ12343456788"; // request unique ID
long expireTime = 1000; // 1000 ms expiration
long expireAt = System.currentTimeMillis() + expireTime;
String val = "expireAt@" + expireAt;
Boolean firstSet = stringRedisTemplate.execute((RedisCallback
) connection ->
    connection.set(KEY.getBytes(), val.getBytes(), Expiration.milliseconds(expireTime), RedisStringCommands.SetOption.SET_IF_ABSENT));
boolean isConsiderDup = (firstSet == null || !firstSet);

When a request does not contain a unique ID, the article proposes building one from business parameters. By concatenating user ID, method name, and a hash of the request payload, a deterministic key can be generated. The payload (usually JSON) is sorted, concatenated, and an MD5 digest is taken:

String KEY = "dedup:U=" + userId + "M=" + method + "P=" + reqParamMD5;

Because some fields (e.g., timestamps or GPS coordinates) change between identical logical requests, they should be excluded before hashing. An example JSON with a requestTime field demonstrates this:

// two requests differ only by requestTime
String req = "{\n" +
    "\"requestTime\" :\"20190101120001\",\n" +
    "\"requestValue\" :\"1000\",\n" +
    "\"requestKey\" :\"key\"\n}";
String req2 = "{\n" +
    "\"requestTime\" :\"20190101120002\",\n" +
    "\"requestValue\" :\"1000\",\n" +
    "\"requestKey\" :\"key\"\n}";

A reusable helper class ReqDedupHelper is provided to parse a JSON string into a TreeMap , remove excluded keys, serialize back to JSON, and compute the MD5 digest:

public class ReqDedupHelper {
    public String dedupParamMD5(final String reqJSON, String... excludeKeys) {
        TreeMap paramTreeMap = JSON.parseObject(reqJSON, TreeMap.class);
        if (excludeKeys != null) {
            List
excludeList = Arrays.asList(excludeKeys);
            for (String key : excludeList) {
                paramTreeMap.remove(key);
            }
        }
        String json = JSON.toJSONString(paramTreeMap);
        return jdkMD5(json);
    }
    private static String jdkMD5(String src) {
        try {
            MessageDigest md = MessageDigest.getInstance("MD5");
            return DatatypeConverter.printHexBinary(md.digest(src.getBytes()));
        } catch (Exception e) { return null; }
    }
}

The article includes a main method that demonstrates the helper: without excluding requestTime the two requests produce different MD5 values; after exclusion they produce identical values, confirming the deduplication logic.

Finally, the complete solution combines the parameter‑MD5 key with Redis SETNX+expiration to achieve atomic deduplication within a configurable time window, as shown in the concluding code snippet.