Elasticsearch as a Time Series Engine: Practices, Challenges, and Alibaba Cloud TimeStream Solutions

Elasticsearch, originally a general‑purpose search engine, is being extended to serve as a time‑series engine for monitoring and IoT scenarios.

Reasons include a collaboration between the ES community and Alibaba Cloud to address metric storage pain points, the need for high‑TPS writes, a fixed data model (timestamp, dimension, metric), vertical‑write horizontal‑read pattern, and the stable nature of sampled data without peaks.

Key challenges are a steep learning curve, complex DSL queries compared with PromQL, slower query performance, and excessive storage cost—ES can require up to 74× the storage of InfluxDB for the same data.

Alibaba Cloud’s Elasticsearch TimeStream addresses these issues with several capabilities:

Setting index.mode=time_series and mapping fields time_series_dimension and time_series_metric to generate an internal _tsid and automatic @timestamp .

Columnar storage and LSM‑like data structures for high‑throughput writes.

Source compression (synthetic _source ) to reduce metadata overhead.

TimeSeries DataStream (TSDS) with index.time_series.start_time and index.time_series.end_time for elastic time‑partitioning and seamless rollover.

TimeStream also provides built‑in down‑sampling, allowing data to be stored at multiple granularities (1 min, 10 min, 60 min), which cuts storage by up to 90 % and speeds up queries.

Integration with the observability stack is supported: TimeStream implements Prometheus remote‑write, enabling direct ingestion from Prometheus, and can be queried from Grafana using the Prometheus data source, offering full PromQL compatibility.

Benchmarks show that after compression and metadata removal, ES storage approaches that of InfluxDB, and query performance on down‑sampled data can surpass both InfluxDB and ClickHouse in many cases.

In summary, the Elasticsearch time‑series engine, enhanced by TimeStream, provides a distributed, highly available, and feature‑rich solution for metric storage and analysis, complementing the broader Elastic Stack ecosystem.