QTSDB: A Distributed Time‑Series Database – Architecture, Data Access, and Fault Handling

QTSDB is a distributed time‑series database developed by the 360 infrastructure team to overcome the single‑node limitations of open‑source InfluxDB, providing capacity expansion, replica fault‑tolerance, and high‑throughput data ingestion.

Key features include a high‑performance storage engine optimized for time‑series data, SQL‑like query language with aggregation functions, automatic expiration of old data, built‑in continuous queries, and a Go implementation with no external dependencies.

The system is divided into three components: a stateless proxy layer, a meta cluster that stores logical storage hierarchy and mapping information using Raft for strong consistency, and a data cluster where each shard corresponds to a TSM storage engine.

Data is organized hierarchically: databases contain retention policies, which contain shard groups, which are further split into shards distributed across physical nodes. Writes are routed to shards based on a hashed series key, and replicas are written using a leader‑less multi‑write strategy.

Read requests first locate the appropriate database, retention policy, and shard group via the meta service, then map shards to physical nodes. Queries are translated into storage keys by constructing inverted indexes (TSI) that map measurements, tags, and fields to series keys.

Both non‑aggregated and aggregated queries are processed by retrieving data from relevant shards, merging results at the data node level, and performing final aggregation at the proxy before returning results to the client.

Fault handling relies on replica redundancy; if a data node fails, writes are temporarily stored by the proxy and replayed once the node recovers. The meta service monitors node health and selects online replicas for reads.

When new nodes join the data cluster, a new shard group is created to immediately distribute incoming writes across all nodes, while existing data remains accessible without migration.

In summary, QTSDB implements a two‑stage query flow—shard‑level processing followed by proxy‑level aggregation—offering scalable, fault‑tolerant time‑series storage suitable for large‑scale data ingestion scenarios.