Elasticsearch Cluster Architecture and Deployment Strategies

Elasticsearch Cluster Architecture

Elasticsearch is a widely used open‑source search and analytics system that serves search, JSON document storage, and time‑series analysis.

Key Concepts

Node: a running Elasticsearch instance (process) on a machine.

Index: logical collection of documents, consisting of mappings and inverted index files.

Shard: a partition of an index; each shard is managed by a node. Primary and replica shards exist.

Replica: copy of a shard that provides high availability and read scalability.

Indexing flow: a document is routed to the primary shard, indexed there, then replicated to its replica shards before the operation returns.

Deployment Modes

Two main role‑deployment patterns are described:

Mixed deployment : Data and transport roles share the same node. Simple to start but can cause resource contention and connection limits.

Layered deployment : Separate Transport nodes (routing/merging) from Data nodes (storage/processing). Improves isolation, scalability, and supports hot upgrades.

Data Layer Architecture

Elasticsearch stores index and metadata on local file systems using various loading methods (niofs, mmap, etc.). Replicas ensure reliability and query capacity.

Two broader distributed‑data system architectures are compared:

Local‑file‑system based clusters with primary/replica shards; recovery requires copying data between nodes.

Shared‑storage clusters where compute nodes access a distributed file system, allowing faster node replacement and independent scaling of storage and compute.

Summary

Both architectures have trade‑offs; mixed deployment is easy to use, while layered deployment offers better isolation and scalability. Choosing the appropriate model depends on workload characteristics and operational requirements.