Why Loki Beats ELK for Cloud‑Native Log Management

Background and Motivation

When an application or a node in a container cloud encounters problems, the typical troubleshooting flow involves checking metrics and alerts from Prometheus, then manually inspecting pod logs from stdout/stderr. Without a centralized log system, this process is cumbersome and slows incident response.

Our monitoring is built on Prometheus, where metrics indicate when a threshold is crossed and trigger alerts, but metrics alone do not reveal the root cause. Kubernetes pods write logs to stdout/stderr, and operators must retrieve these logs to diagnose issues such as sudden memory spikes.

Without a log aggregation system, operators have to switch between Grafana for metrics and Kibana for logs, which degrades user experience. Loki’s primary goal is to minimize the cost of switching between metrics and logs, thereby reducing incident response time and improving user experience.

Problems with ELK

Full‑text indexing solutions like ELK provide rich features but are heavyweight, consume many resources, and involve complex operations. Most queries only need a limited time range and simple parameters (e.g., host, service), making ELK feel like using a sledgehammer for a nail.

Loki aims to strike a balance between query simplicity and expressive power.

Cost

Full‑text search incurs high indexing costs due to inverted index creation and storage. Alternative designs such as OKlog use mesh‑based distribution and eventual consistency to lower cost, though they sacrifice query convenience. Loki’s third goal is to provide a cost‑effective logging solution.

Overall Architecture

Loki’s architecture mirrors Prometheus by using the same label set as an index, allowing log queries to be performed alongside metric queries. This reduces the need for separate indexing pipelines and cuts storage overhead.

Promtail runs as a DaemonSet on each Kubernetes node, discovers logs via the Kubernetes API, attaches proper metadata, and forwards them to Loki. The storage architecture is shown below.

Read/Write Path

Distributor

The Distributor receives logs from promtail, batches them, and compresses them (gzip) before handing them off to Ingester instances. This prevents overwhelming the storage backend with raw write traffic.

Chunks are replicated (default three times) for redundancy and resilience.

Ingester

Ingester builds compressed chunks from incoming logs. When a chunk reaches a size or time threshold, it flushes the chunk to the storage backend. Separate databases store chunks and indexes because they contain different data types.

After flushing, the Ingester creates a new empty chunk for subsequent log entries.

Querier

Querier handles read requests by taking a time range and label selector, consulting the index to find matching chunks, and performing distributed greps to return results. It also pulls the latest unflushed data from Ingester instances.

Scalability

Loki’s index can be stored in Cassandra, Bigtable, or DynamoDB, while chunks can reside in various object stores. Distributor and Querier are stateless; Ingester is stateful but rebalances chunks when nodes are added or removed. The underlying storage layer, Cortex, has been proven in production for years, giving confidence to experiment with Loki in real environments.