Redis vs Dragonfly: Benchmark Comparison and Architectural Insights

1. Redis Blog Article Translation

In mid‑2023 a former Google and Amazon engineer released an open‑source memory data cache system called Dragonfly , written in C/C++ and distributed under the Business Source License (BSL). Benchmark results suggest it may be the fastest in‑memory storage system, supporting Memcached and Redis protocols with higher query performance and lower runtime memory consumption.

Compared with Redis, Dragonfly claims a 25× performance boost under typical workloads, can handle millions of requests per second per server, and uses about 30% less memory in a 5 GB storage test.

Within two months of release, Dragonfly earned 9.2K GitHub stars and 177 forks. Although other Redis‑compatible memory stores such as KeyDB and Skytable exist, none have generated comparable buzz, as a clean‑slate design can discard legacy constraints.

2. Speed Issue

The original benchmark compared a single‑process Redis instance (single core) with a multi‑threaded Dragonfly instance (all available cores). To make the comparison fair, Redis 7.0 was tested as a 40‑shard cluster on AWS c4gn.16xlarge instances, matching the hardware used for Dragonfly .

In this setup Redis’s throughput exceeded Dragonfly by 18%‑40% while using only 40 of the 64 vCores.

3. Architectural Differences

Background : Multi‑threaded projects often arise from pain points in single‑process designs. Redis originally did not target many‑core utilization, but many vendors have added solutions.

Redis achieves horizontal scalability via clustering and Redis Enterprise, which provides management, high availability, failover, persistence, and backup.

Design Principles :

Run multiple Redis instances per VM to achieve linear vertical and horizontal scaling without shared resources.

Limit each Redis process size (≤25 GB, ≤50 GB with Redis on Flash) to ease replication, snapshotting, and AOF rewriting.

Prioritize horizontal scaling for elasticity, cost‑effectiveness, higher throughput, NUMA friendliness, and to avoid storage‑throughput bottlenecks.

Horizontal scaling also reduces impact of node failures and simplifies capacity expansion compared with vertical scaling.

4. Summary

Community‑driven ideas such as io_uring , modern dictionaries, and more strategic threading may be incorporated into Redis in the future, but Redis will retain its core principles of multi‑process, no‑shared‑memory architecture for optimal performance, scalability, and resilience.

Appendix: Redis 7.0 vs Dragonfly Benchmark Details

1) Result Overview

Redis 7.0.0 was built from source; Dragonfly used the June 3 source (hash =e806e6c…). The goal was to reproduce Dragonfly’s published results and verify them under identical memtier_benchmark and OS conditions.

2) Analysis Results

Single GET latency < 1 ms : Redis achieved 4.43 M ops/s (0.383 ms avg latency); reproduced Dragonfly reached 3.80 M ops/s (0.390 ms avg).

30‑parallel GET : Redis 22.9 M ops/s (2.239 ms avg); reproduced Dragonfly 15.9 M ops/s (3.99 ms avg). Redis outperformed by 43%‑52%.

Single SET latency < 1 ms : Redis 4.74 M ops/s (0.391 ms avg); reproduced Dragonfly 4.00 M ops/s (0.500 ms avg). Redis ahead by ~19%.

30‑parallel SET : Redis 19.85 M ops/s (2.879 ms avg); reproduced Dragonfly 14.0 M ops/s (4.203 ms avg). Redis ahead by 42%‑99%.

3) memtier_benchmark Commands

Redis:2X: memtier_benchmark –ratio 0:1 -t 24 -c 1 –test-time 180 –distinct-client-seed -d 256 –cluster-mode -s 10.3.1.88 –port 30001 –key-maximum 1000000 –hide-histogram

Dragonfly: memtier_benchmark –ratio 0:1 -t 55 -c 30 -n 200000 –distinct-client-seed -d 256 -s 10.3.1.6 –key-maximum 1000000 –hide-histogram

Similar commands were used for 30‑parallel GET/SET pipelines.

4) Test Environment

VM type: AWS c6gn.16xlarge

Architecture: aarch64, Arm Neoverse‑N1, 64 cores per socket, 1 NUMA node

Kernel: 5.10, Memory: 126 GB

References: Redis Architecture 13 Years Later , Reddit discussion .