Understanding Loki: Advantages, Architecture, Installation, and Query Practices

Loki differs from Elasticsearch by indexing only log labels, dramatically reducing index resource overhead and enabling low‑cost, high‑performance queries.

It achieves concurrent querying by splitting queries into small shards, similar to parallel grep , and aligns its label model with Prometheus, allowing seamless integration with Alertmanager.

The system consists of four roles: querier (query processor), ingester (log storage), query‑frontend (pre‑query dispatcher), and distributor (write distributor), selectable via the -target binary flag.

Read path: The querier receives HTTP requests, forwards them to ingesters for in‑memory data, falls back to storage if needed, deduplicates results, and returns the final dataset over HTTP.

Write path: The distributor receives HTTP write requests, hashes streams to route them to appropriate ingesters (based on replication factor), creates or appends chunks per tenant and label set, and acknowledges success.

Installation (local mode):

$ wget https://github.com/grafana/loki/releases/download/v2.2.1/loki-linux-amd64.zip
$ wget https://github.com/grafana/loki/releases/download/v2.2.1/promtail-linux-amd64.zip

Configure Promtail:

mkdir -p /opt/app/{promtail,loki}
cat <
/opt/app/promtail/promtail.yaml
server:
  http_listen_port: 9080
  grpc_listen_port: 0
positions:
  filename: /var/log/positions.yaml
client:
  url: http://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name: system
  static_configs:
  - targets: [localhost]
    labels:
      job: varlogs
      __path__: /var/log/*.log
EOF
EOF

Create a systemd service for Promtail and start it.

Configure Loki:

cat <
/opt/app/loki/loki.yaml
auth_enabled: false
server:
  http_listen_port: 3100
  grpc_listen_port: 9096
ingester:
  wal:
    enabled: true
    dir: /opt/app/loki/wal
# ... (additional configuration omitted for brevity) ...
EOF
EOF

Create a systemd service for Loki and start it.

In Grafana, add Loki as a data source and use the Explore UI to query logs, e.g., rate({job="message"} |= "kubelet" [1m]) .

Label‑only indexing means queries use Prometheus‑style label matchers, such as {job="syslog"} or regex matchers like {job=~"apache|syslog"} . Dynamic tags can be extracted via regex stages in Promtail, enabling streams like action=get or status_code=200 .

High‑cardinality labels (e.g., per‑IP) can explode the number of streams and degrade performance; avoid using such labels unless necessary.

When fields are not indexed as labels, queries can still filter them using pipeline expressions, e.g., {job="apache"} |= "11.11.11.11" , but indexing by label is more efficient.

Loki shards queries by time range, allowing configurable parallelism; large deployments can run dozens of queriers to handle terabytes of logs.

Compared to Elasticsearch’s always‑on large index, Loki’s on‑demand sharding reduces memory pressure and improves scalability.

Best practices include adding tags only when they improve query selectivity, monitoring chunk sizes (e.g., chunk_target_size=1MB ), and ensuring logs are ingested in chronological order to avoid rejection of old data.