Design and Implementation of a Redis-Based Delayed Queue Service

Background: In business processes, scenarios such as unpaid orders after 30 minutes, missing user reviews, or undelivered orders require delayed handling; polling tables works for small data but becomes resource‑intensive at scale.

Types of delayed queues: Java's java.util.concurrent.DelayQueue (simple but in‑JVM only), RocketMQ delayed queue (persistent and distributed but limited granularity), RabbitMQ delayed queue using TTL+DLX (persistent and distributed but same‑delay messages share a queue).

Key considerations for building a custom delayed‑queue service include message storage, real‑time retrieval of expired messages, and high availability.

Redis‑based implementation (version 1.0) stores messages in a hash (Message Pool), uses 16 ZSETs as delayed queues (score = expiration time), and a timed task scans queues. Message fields include tags, keys, body, delayTime, and expectDate.

Version 2.0 improves latency by replacing the 1‑minute polling task with a Java Lock await/signal mechanism, enabling real‑time delivery of expired messages.

Multi‑node deployment adds pull‑job threads per queue, worker threads for processing, and Zookeeper coordination for queue rebalancing; pull jobs track offsets to avoid message loss on failures, and services register with Zookeeper on startup.

The main workflow: on service start, register with Zookeeper, obtain assigned queues, launch pull‑job threads, query for expired messages, hand them to workers, and remove them after successful processing; offsets are adjusted to handle retries.