CB‑SQL Overview: Architecture, RocksDB Foundations, and Performance Optimizations

CB‑SQL is a new generation elastic database developed by JD.com’s Database Technology team, compatible with the MySQL protocol and based on the open‑source distributed database CockroachDB. It adopts a layered architecture inspired by Google Spanner, consisting of an SQL layer that maps relational concepts (databases, tables, schemas, rows, columns, indexes) to operations on a distributed KV layer built on RocksDB.

RocksDB, originally created by Facebook as a C++ embedded KV store, uses a Log‑Structured Merge‑Tree (LSM) architecture with three components: Memtable (in‑memory write buffer), Log (WAL for durability), and SSTable (immutable on‑disk sorted files). Reads follow the order Active Memtable → Immutable Memtable → SSTable, and RocksDB employs multi‑level compaction (default seven levels) to reduce read amplification.

The article provides a comprehensive RocksDB tuning guide, covering DB‑level parameters such as write_buffer_size , use_fsync , total_threads , optimize_filters_for_hits , num_levels , and various level‑specific settings (e.g., lru_cache_capacity , bloom_filter_bits_per_key , block_size ). It also discusses table‑level options like cache configuration and compression algorithms per level.

CB‑SQL integrates RocksDB via CGO in Golang, optimizing away costly CGO calls by batching writes, pushing MVCC logic to C++, and minimizing memory copies. Write paths use a custom Batch that shares the same encoding as RocksDB’s WriteBatch, allowing direct memory‑segment exchange between Go and C++. Read paths avoid frequent Get calls by leveraging MVCC iterators and selective value fetching.

For large‑scale deletions, CB‑SQL runs its own compaction thread to trigger proactive compactions, balancing space reclamation with overall performance. It also exploits RocksDB’s Ingest feature to import pre‑built SST files, enabling efficient bulk data import and high‑throughput backup/restore operations (e.g., exporting SSTs to S3 and re‑ingesting them).

Overall, the article presents the architectural choices, key RocksDB concepts, and practical parameter adjustments that make CB‑SQL a high‑performance, scalable storage engine for JD.com’s distributed services.