Redis vs Dragonfly: Benchmark Comparison and Architectural Insights

In mid‑2023 a former Google and Amazon engineer released the open‑source memory‑data cache system Dragonfly , written in C/C++ and distributed under the Business Source License. Dragonfly claims to be the world’s fastest in‑memory storage system, supporting both Memcached and Redis protocols while offering higher query performance and lower runtime memory consumption.

Dragonfly Performance Claims

According to the original benchmarks, Dragonfly can achieve up to 25× performance improvement over Redis under typical workloads, handle millions of requests per second on a single server, and use 30% less memory than Redis in a 5 GB storage test.

Redis Response

Redis co‑founder and CTO Yiftach Shoolman, along with Redis Labs architects, published an article titled “After 13 years, does Redis need a new architecture?” providing their own Redis 7.0 vs Dragonfly benchmark results. They report that Redis’s throughput exceeds Dragonfly’s by 18%–40% and argue that the original Dragonfly benchmarks do not reflect real‑world usage.

Speed Comparison

Redis’s tests compare a 40‑shard Redis 7.0 cluster (using most of the available cores) against the largest Dragonfly instance (AWS c4gn.16xlarge). Results show Redis achieving higher throughput in both single‑threaded and 30‑threaded scenarios, with latency remaining below 1 ms for GET and SET operations.

Architectural Differences

Redis emphasizes a multi‑process, no‑shared‑memory design that scales horizontally by running multiple instances per VM, enabling better fault tolerance, replication speed, and resource utilization. It also enforces size limits on individual processes (≤25 GB, or 50 GB with Redis on Flash) to keep memory overhead low and simplify cluster management.

Key architectural principles highlighted include:

Running multiple Redis processes per VM for linear vertical and horizontal scaling.

Limiting each process size to reduce memory overhead during fork‑based operations.

Prioritizing horizontal scaling for elasticity, cost‑effectiveness, high throughput, and NUMA friendliness.

Conclusion

While Redis acknowledges the innovative ideas introduced by Dragonfly, it maintains confidence in its own architecture as the best solution for real‑time in‑memory data platforms, citing superior performance, scalability, and resilience.

Benchmark Details

Versions Used

Redis 7.0.0 built from source.

Dragonfly built from the June 3 source snapshot (hash e806e6c…).

Test Goals

Validate Dragonfly’s published results and reproduce them.

Determine the best achievable performance of an OSS Redis 7.0 cluster on AWS c6gn.16xlarge and compare it with Dragonfly.

Client Configuration

Heavy use of open connections for memtier_benchmark threads to all shards.

Two memtier_benchmark processes on the same client VM yielded the best results.

Resource Utilization

Redis performed best with 40 primary shards (24 spare vCPUs) without full CPU utilization.

Dragonfly fully saturated all 64 vCPUs.

Benchmark Commands

Redis: memtier_benchmark –ratio 0:1 -t 24 -c 1 –test-time 180 –distinct-client-seed -d 256 –cluster-mode -s 10.3.1.88 –port 30001 –key-maximum 1000000 –hide-histogram

Dragonfly: memtier_benchmark –ratio 0:1 -t 55 -c 30 -n 200000 –distinct-client-seed -d 256 -s 10.3.1.6 –key-maximum 1000000 –hide-histogram

Similar commands with –pipeline 30 were used for the 30‑threaded GET/SET tests.

Test Environment

Both client and server VMs were AWS c6gn.16xlarge instances (aarch64, ARM Neoverse‑N1) with 126 GB RAM, 64 cores, and a single NUMA node.

References: Redis Architecture Blog , Reddit discussion .