Vivo Database and Storage Platform: Architecture, Capabilities, and Future Roadmap

The article is a written summary of Xiao Bo’s talk at the 2021 Vivo Developer Conference, describing the background, construction, capabilities, and future plans of Vivo’s self‑built database and storage platform.

Background and growth

Vivo’s internet business grew from 2.2 billion mobile users in 2018 to 2.7 billion total users in 2021, with dozens of applications exceeding 100 million monthly active users. Correspondingly, the number of database instances grew nearly five‑fold, server count increased 6.8 times, and DBA‑per‑person workload rose 14.9 times.

Problems in 2018

Low availability caused by inefficient SQL, manual operations, poor architecture, and unstable open‑source components.

Unstandardized, command‑line‑only change processes.

High operational cost.

Insufficient security (lack of data classification, password policies, etc.).

Platform capability construction

The platform is organized in two layers. The first layer provides the core database and storage products (relational, NoSQL, and storage services). The second layer offers tooling: unified management platforms, data‑transfer services, white‑screen operation tools, SQL audit/optimization, backup, and other utilities.

Tools are mainly self‑developed, while the underlying database/storage products are selected from mature open‑source solutions or built in‑house.

DaaS (Database‑as‑a‑Service)

DaaS delivers a highly self‑service, intelligent, highly available, low‑cost data storage solution covering the full lifecycle from request to decommission. Its four value propositions are:

Improved availability through inspection, monitoring, pre‑plans, and incident post‑mortems.

Enhanced development efficiency via self‑service provisioning, change detection, and optimization diagnostics.

Strengthened data security via permission control, encryption, masking, audit, and backup encryption.

Reduced operational cost through automation, resource orchestration, and efficient utilization.

The platform now supports six database products (MySQL, Elasticsearch, MongoDB, TiDB, etc.) with capabilities such as pre‑change SQL review, backup, one‑click rollback, and audit tracing.

MySQL evolution

MySQL started with a 1.0 version built on MHA and custom components. The current 2.0 version removes MHA, introduces a self‑developed proxy layer fully compatible with MySQL protocol, providing three‑level read/write separation, traffic control, transparent encryption, SQL firewall, and logging. The proxy works with high‑availability components and RAFT for internal control. Future 3.0 plans include multi‑region active‑active deployment.

Redis enhancements

Vivo migrated from single‑node and master‑slave Redis to a clustered mode with automatic failover and elastic scaling. For TB‑scale clusters, Vivo added:

Multi‑active Redis for multi‑datacenter availability.

Improved persistence (AOF redesign, AEP hardware, fork‑less RDB).

Performance upgrades (asynchronous replication, file cache, water‑level control, command‑complexity optimization).

To address storage beyond memory, a disk‑based KV store built on TiKV was created, exposing the Redis protocol for seamless migration.

Object & file storage

The service offers EB‑scale capacity with both object‑storage and POSIX file interfaces, supporting AI training, HPC, and billions of small files. It also provides built‑in image/video processing (watermark, thumbnail, transcoding).

Security

Passwords are stored encrypted and decrypted via a centralized key‑management system. Sensitive data is automatically classified, labeled, and protected with access controls, digital watermarks, and transparent encryption at rest. Backup and log data are also encrypted.

Change management

Vivo implements non‑locking schema and data changes with three safety checkpoints (pre‑, during, post‑deployment) and one‑click rollback. A one‑click cross‑cluster synchronization and GUI design platform enable 24/7 self‑service data changes for developers.

Cost management

Budget control and resource‑level forecasting.

Standardized deployment packages to improve utilization.

Hybrid deployment of database proxies and object‑storage nodes for CPU‑storage complementarity.

Continuous capacity monitoring, scaling, and decommissioning of idle clusters.

Future roadmap

Vivo plans to expand the storage service matrix, introduce more SaaS offerings, and further develop autonomous services (performance, capacity, intelligent diagnostics). Data security will receive additional investment, and the whole stack will be integrated into Vivo’s hybrid‑cloud architecture to provide a unified, standardized experience.