Handling Duplicate Requests in Backend Services with Redis and Java

Duplicate requests can cause serious issues such as repeated orders in transactional systems; while read‑only queries may tolerate them, write operations must be guarded. The article discusses server‑side strategies for uniformly handling duplicate requests, excluding client‑side click‑prevention.

01. Use a Unique Request Identifier for Deduplication

By assigning each request a unique ID and storing it in Redis, any subsequent request with the same ID can be considered a duplicate if the key still exists.

String KEY = "REQ12343456788"; // request unique ID
long expireTime = 1000; // 1000 ms expiration, duplicate within this window
long expireAt = System.currentTimeMillis() + expireTime;
String val = "expireAt@" + expireAt;
Boolean firstSet = stringRedisTemplate.execute((RedisCallback
) connection ->
    connection.set(KEY.getBytes(), val.getBytes(), Expiration.milliseconds(expireTime), RedisStringCommands.SetOption.SET_IF_ABSENT));
boolean isConsiderDup;
if (firstSet != null && firstSet) {
    isConsiderDup = false; // first access
} else {
    isConsiderDup = true; // key already exists, duplicate
}

02. Deduplicate Based on Business Parameters

If requests lack a unique ID, the request parameters themselves can serve as identifiers. For a simple case with a single parameter reqParam , a composite key like userId:method:reqParam can detect duplicates.

String KEY = "dedup:U=" + userId + "M=" + method + "P=" + reqParam;

03. Compute a Digest of Request Parameters

When parameters are JSON objects, sort the keys, concatenate them, and compute an MD5 hash to use as a compact identifier. This reduces key length while preserving uniqueness.

String KEY = "dedup:U=" + userId + "M=" + method + "P=" + reqParamMD5;

04. Optimize by Removing Time‑Sensitive Fields

Requests often contain timestamps or GPS data that differ slightly between calls. Excluding such fields before hashing prevents false negatives in deduplication.

05. Java Utility Class for Request Deduplication

public class ReqDedupHelper {
    /**
     * Compute MD5 of request JSON after removing specified keys.
     */
    public String dedupParamMD5(final String reqJSON, String... excludeKeys) {
        String decrptParam = reqJSON;
        TreeMap paramTreeMap = JSON.parseObject(decrptParam, TreeMap.class);
        if (excludeKeys != null) {
            List
dedupExcludeKeys = Arrays.asList(excludeKeys);
            if (!dedupExcludeKeys.isEmpty()) {
                for (String dedupExcludeKey : dedupExcludeKeys) {
                    paramTreeMap.remove(dedupExcludeKey);
                }
            }
        }
        String paramTreeMapJSON = JSON.toJSONString(paramTreeMap);
        String md5deDupParam = jdkMD5(paramTreeMapJSON);
        log.debug("md5deDupParam = {}, excludeKeys = {} {}", md5deDupParam, Arrays.deepToString(excludeKeys), paramTreeMapJSON);
        return md5deDupParam;
    }
    private static String jdkMD5(String src) {
        try {
            MessageDigest md = MessageDigest.getInstance("MD5");
            byte[] mdBytes = md.digest(src.getBytes());
            return DatatypeConverter.printHexBinary(mdBytes);
        } catch (Exception e) {
            log.error("", e);
            return null;
        }
    }
}

Test logs demonstrate that when the requestTime field is excluded, two otherwise identical requests produce the same MD5, confirming successful deduplication.

06. Summary

The complete solution combines a unique request ID (or a hashed parameter set), removal of volatile fields, and an atomic Redis SETNX with expiration to reliably block duplicate requests within a short time window.