Understanding Redis: Core Concepts, Persistence, Replication, Sentinel Failover, and Cluster Sharding

Redis is an open‑source, in‑memory key‑value database implemented in C. It supports five primary data types—string, list, hash, set, and sorted set—each backed by one or two efficient internal structures such as simple dynamic strings, linked lists, ziplists, hash tables, skip lists, and integer arrays.

Key Storage and Rehashing

All keys are stored in a hash table where each bucket points to an entry containing pointers to the actual key and value. Redis achieves O(1) lookups by hashing the key, but hash collisions can occur. To mitigate collisions Redis performs a rehash that creates a larger hash table and gradually migrates entries, using a progressive rehash algorithm that moves a few buckets per client request to avoid blocking the main thread.

Single‑Threaded Execution Model

Redis processes network I/O and command execution in a single thread, eliminating context switches and lock contention. High concurrency is achieved through I/O multiplexing (select/epoll). Since Redis 6, optional I/O threads handle network reads/writes, while command processing remains single‑threaded.

Persistence Mechanisms

Redis offers two main persistence strategies:

AOF (Append‑Only File) : Logs every write command. Three appendfsync policies— always (maximum safety, lowest performance), everysec (balanced), and no (fastest, riskier). AOF files can be rewritten to compact redundant commands, performed by a background bgrewriteaof process that forks to avoid blocking.

RDB (Snapshot) : Periodically creates a point‑in‑time dump of the dataset. The save command blocks the server, while bgsave forks a child process to write the snapshot without blocking the main thread. RDB provides fast recovery but may lose recent writes.

Hybrid usage combines infrequent RDB snapshots with AOF logging of intermediate writes, giving fast recovery and minimal data loss.

Replication

Redis uses a master‑slave model. Slaves synchronize via the PSYNC command, receiving a full RDB dump followed by a replication buffer that streams writes occurring after the snapshot. The master’s bgsave creates the snapshot, and the buffer ensures no writes are lost during sync. Cascading replication (master → intermediate slave → downstream slaves) reduces load on the master.

Sentinel Failover

Sentinel processes continuously ping masters and slaves. When a majority (quorum) of Sentinels agree a master is objectively down, they elect a leader to select a new master based on slave priority, replication offset, and instance ID. The chosen master is announced to all slaves and clients, and the failover proceeds automatically.

Redis Cluster and Sharding

Redis Cluster partitions data across up to 16384 hash slots. A key’s CRC16 hash modulo 16384 determines its slot. Slots are assigned to cluster nodes, and clients learn the slot‑to‑node map on first connection. If a client contacts the wrong node, the server replies with MOVED (permanent redirection) or ASK (temporary during slot migration). Clients then resend the command to the correct node.

Cluster scaling is achieved by adding or removing nodes and rebalancing slots, allowing horizontal scaling without single‑node bottlenecks.