Database Selection Strategies and iQIYI's Implementation of MySQL, Redis, Couchbase, and HiKV

This article shares practical insights on database selection and the concrete practices adopted by iQIYI for various database systems.

When choosing a database, the first step is to identify who initiates the selection—procurement, DBA, or application developers—because each group focuses on different concerns such as cost, operational overhead, stability, performance, scalability, and security.

Operational cost : monitoring, backup/recovery, upgrade, community health, tuning, troubleshooting.

Stability : multi‑replica, high availability, multi‑write support.

Performance : latency, QPS, advanced storage tiers.

Scalability : horizontal and vertical expansion.

Security : audit compliance, protection against SQL injection and data exfiltration.

iQIYI’s database portfolio includes MySQL, TiDB, Redis, Couchbase, MongoDB, other NoSQL stores, and big‑data analysis systems such as Hive and Impala. The databases are classified by interface (SQL vs. NoSQL) and workload type (OLTP, OLAP, HTAP).

MySQL usage : master‑slave with semi‑synchronous replication, weekly full backups plus daily incremental backups, and performance‑enhanced Xtrabackup that reduces full‑restore time from 5 hours to 100 minutes and enables single‑table recovery. DDL/DML tools (gh‑ost, oak‑online‑alter‑table) are throttled based on replication lag. High availability is achieved with a modified MHA architecture using a master‑agent model and Raft groups for cross‑region failover. Scaling is handled via ShardingSphere SDK and proxy solutions, while an audit plugin streams full SQL logs to Kafka for downstream analysis, employing a two‑level RingBuffer to keep performance impact below 2 %.

Redis usage : master‑slave with Sentinel, customized Sentinel deployment across data centers to avoid split‑brain. A real‑time backup process mirrors data to a KV store (ScyllaDB) to protect against accidental data loss. Automatic alert tuning expands buffers during high‑QPS periods, and an auto‑scaling mechanism adjusts cluster size. The Java client Jedis is optimized to rebuild connections only for failed shards. DNS TTL shortening is replaced by a Redis Name Service (RNS) that fetches topology from Sentinel, allowing direct IP connections and faster failover. A proxy layer writes updates to Kafka for cross‑cluster synchronization.

Couchbase usage : employed as a high‑performance KV store with two bucket types—Memcached (no persistence) and Couchbase (JSON persistence with replicas). Data is hashed to vBuckets; the client’s cluster map updates dynamically during rebalancing, making failover transparent. XDCR provides bi‑directional replication between clusters, and a Java SDK enables manual failover by switching write targets.

HiKV (self‑developed KV store) : built on ScyllaDB’s distributed engine, adding a custom SSD‑backed storage engine where keys reside in memory and values on disk. An in‑memory index with fixed‑size entries and a checkpoint mechanism reduces I/O compared with LSM‑Tree based engines. HiKV now serves ~30 % of iQIYI’s former Couchbase workload, lowering storage costs.

Database operations management evolution : from DBA‑written scripts, to a private‑cloud self‑service portal, to a web‑based UI handling ~90 % of routine tasks, and finally to a toolbox of automated diagnostics, alerting, intelligent scheduling, and resource tagging that empowers developers to resolve issues independently.

Practical selection tree : a decision‑tree guide that maps requirements (data volume, QPS, latency, backup needs, proxy usage, etc.) to suitable database families (relational, NoSQL, HTAP, big‑data). The tree is available on the internal self‑service platform.

Final reflections : emphasize understanding real requirements, avoiding over‑engineering, evaluating technologies based on actual problem‑solving capability, and maintaining an open‑source mindset.