FeatureKV: A High‑Performance, Scalable Key‑Value Store for Billion‑Scale Read/Write Workloads at WeChat

FeatureKV is a high‑performance, strongly consistent key‑value storage system developed within WeChat to meet the massive read/write demands of services such as “Look‑at‑Look”, ads, payments and mini‑programs, which operate at the scale of billions of keys and billions of queries per second.

The system addresses two “billion‑level” challenges: (1) supporting 10^9 reads per second with low latency, and (2) ingesting 10^9 keys per hour from offline batch jobs while providing versioned data and fast roll‑backs.

FeatureKV is built on three core external services: Chubby (metadata store), a user‑side distributed file system (WFS/HDFS) for input data, and its own distributed file system (FKV_WFS) for storing generated tables. The architecture consists of stateless DataSvr instances that perform batch parsing, routing, indexing and write data to FKV_WFS, and stateful KVSvr instances that load tables from FKV_WFS and serve read‑only queries.

Key design highlights include:

Excellent read performance: in‑memory MemTable can deliver tens of millions of QPS; SSD‑backed tables achieve millions of QPS on commodity hardware.

Excellent write performance: the system can write a billion keys (average value size 400 B) within one hour when the underlying file system is fast enough.

Horizontal scalability: read capacity scales by adding Sects (read replicas), storage capacity and additional read throughput scale by adding Roles, and write throughput scales by adding stateless DataSvr instances.

Batch‑write‑friendly interfaces: task‑based write API accepts a path on the user file system, supports incremental or full updates, TTL, and automatic retry/alerting.

Version management: BatchGet guarantees all keys in a request belong to the same version; the system retains recent versions for fast roll‑back and supports history‑based recovery.

Trade‑offs: online writes are not supported and the system provides eventual consistency rather than strong consistency for the offline‑write use case.

For online reads, FeatureKV employs a high‑performance hash table (MemTable) for hot data, libco‑based asynchronous I/O for SSD tables, custom serialization to reduce RPC overhead, and optional half‑precision floating‑point compression for model vectors.

The distributed transaction BatchGet is implemented using copy‑on‑write (COW) semantics; a global synchronized version is maintained across roles, and clients cache the sync version to avoid extra RPCs.

Version rollback is achieved by keeping a rollback‑version flag per table; when rolling back, the system either switches to a previously cached buffer (seconds) or merges recent incremental updates into a new full version (minutes).

Offline write flow is organized as a Map‑Reduce‑like pipeline: DataSvr parses input files, routes keys to 2400 partitions (HashFun % 2400), sorts each partition (IdxTable/BlkTable) and writes them to FKV_WFS. Distributed slicing and sorting enable linear scalability; the pipeline reduces a 1 TB write from >120 min to ~71 min.

FeatureKV is deployed in more than 10 modules with over 270 machines, serving peak workloads of 11 billion features per second, 15 ms 99th‑percentile latency, and 99.999999 % transaction success. It powers critical WeChat services such as personalized ad retrieval, payment risk control, and user‑profile features.

In summary, FeatureKV demonstrates how a purpose‑built, version‑aware key‑value store can satisfy both massive offline batch ingestion and ultra‑low‑latency online read requirements at internet‑scale.