OpenTSDB Architecture, Data Model, Storage Optimizations, and Practical Use Cases

OpenTSDB is a distributed, scalable time‑series database that can handle millions of writes per second with millisecond precision, leveraging HBase’s LSM tree storage engine and HDFS for high‑performance storage.

The system consists of three main components: a Collector that gathers metrics via telnet or HTTP and forwards them to stateless TSD instances; TSD instances that receive data and provide query services; and an HBase cluster that stores the data.

Data points are modeled as metrics , timestamp , tags (key‑value pairs), and a numeric value. Tags are stored using a three‑byte UID mapping (tsdb‑uid table) to reduce rowkey size and storage space.

Storage optimizations include shortening rowkeys, merging columns into a single column to reduce key‑value count, and using bucketed writes to avoid hotspotting in HBase RegionServers.

Case studies show OpenTSDB powering 58’s data‑platform monitoring system and the 58 intelligent monitoring system, handling traffic data, business metrics, and supporting regression and classification models.

A performance problem arose when a metric received a massive number of datapoints with millions of distinct tag combinations, causing RegionServer handlers to block and queries to become extremely slow.

To resolve the issue, the team performed a multi‑step data‑deletion process: create a test tsdb table mirroring the production schema, copy HBase HFiles via distcp , load them with bulkload , enable delete‑on‑query with tsd.http.query.allow_delete=true , delete the unnecessary data in the test environment, copy the cleaned HFiles back to the online cluster, and finally run a major compaction to restore query performance.

The article also suggests future improvements such as storing tags as metrics to avoid high‑cardinality queries, and concludes with a brief recruitment notice for big‑data engineers.