ScyllaDB Architecture and Performance Optimizations: Design Insights

ScyllaDB is a Cassandra‑compatible NoSQL system that can directly replace Cassandra, delivering more than ten times the throughput on the same hardware while also reducing response latency.

Optimization goals: Efficiency – make the most out of every cycle; Utilization – squeeze every cycle from the machine; Control – spend cycles on what we want, when we want.

Asynchrony everywhere: one thread per core (non‑blocking), asynchronous network I/O, asynchronous file I/O, asynchronous multi‑core processing, with inter‑thread communication via message passing that avoids locks, mutexes, and atomic operations.

The basic performance formula used is Concurrency = Throughput × Latency .

I/O scheduler: The Linux I/O scheduler is disabled because it operates per thread, while ScyllaDB runs only a few threads. A user‑space scheduler assigns different priorities to user requests and background tasks and ensures optimal disk concurrency. During installation, an automatic benchmark increases concurrency until throughput plateaus.

Cache design: ScyllaDB implements its own unified cache instead of the kernel page cache, avoiding 4 KB granularity, unnecessary thread‑safety overhead, synchronous read/write/mmap APIs, and the lack of direct cache‑control interfaces. This unified cache eliminates the need for separate key, row, or page caches.

Self‑tuning features: Like Cassandra, ScyllaDB uses an LSM‑Tree storage structure. It employs an adaptive scheduling strategy that balances CPU, SSD, and network resources, dynamically adjusting background task and request priorities based on memtable write speed and compaction demand.

Memory allocator: The thread‑per‑core design allows ScyllaDB to use a custom memory allocator instead of the system allocator, tracking object types and reference locations. The allocator can move memory blocks when needed to mitigate fragmentation, though this involves trade‑offs.

User‑space TCP/IP (in progress): To avoid the kernel’s heavyweight TCP/IP stack, ScyllaDB uses DPDK to drive hardware directly and implements its own user‑space TCP/IP protocol.

Query processing (in progress): ScyllaDB compiles CQL (Cassandra Query Language) with an LLVM‑JIT, embedding schema and internal layout information into the query execution path to eliminate runtime schema lookups, similar to OceanBase.

ScyllaDB’s code is open source and worth a detailed study. For more information, see the reports at http://t.cn/RvbcUsa and http://t.cn/Ryis2gB .