A Comprehensive Overview of Apache ZooKeeper: Concepts, Architecture, and Practical Usage

Distributed systems consist of hardware or software components spread across multiple networked computers that communicate solely through message passing, introducing challenges such as distributed transactions and data consistency, which have led to classic theories like ACID, CAP, and BASE.

Apache ZooKeeper is an open‑source coordination service that provides highly reliable distributed synchronization, configuration, and naming services. Originating from a Yahoo research project, it was donated to the Apache Software Foundation and now serves as a top‑level project.

ZooKeeper achieves consistency using the ZAB protocol, which is based on the Paxos algorithm. It follows the observer pattern: clients register watches on ZNodes, and any change triggers a one‑time notification to the interested clients.

Key features include a leader‑follower cluster architecture, global data consistency (all servers hold identical copies), strict ordering of client requests, atomic transaction processing, real‑time read guarantees, and durability of committed changes.

Design goals focus on a simple hierarchical data model (a tree of ZNodes), the ability to form clusters, sequential access via globally unique incrementing IDs, and high performance by storing the entire dataset in memory, achieving read‑heavy workloads of 120‑130k QPS.

Each ZNode can store up to 1 MB of data and is identified by a Unix‑style path (e.g., /Configuration/B ). ZNodes can be persistent, ephemeral (tied to a client session), or sequential (auto‑appended numeric suffix).

Typical use cases include a unified naming service (e.g., Dubbo uses ZooKeeper for service discovery), configuration publishing/subscribing, cluster management and master election, soft load balancing, distributed coordination via watches, and distributed locks implemented with either exclusive nodes or ordered sequential nodes.

To get started locally, install JDK, download and extract the ZooKeeper binary (e.g., tar -zxvf apache-zookeeper-3.5.7-bin.tar.gz ), rename zoo_sample.cfg to zoo.cfg , set dataDir in the config, then start the server with bin/zkServer.sh start . Verify the process with jps and check status via bin/zkServer.sh status . Use the client bin/zkCli.sh to interact, and exit with quit . Stop the server using bin/zkServer.sh stop .

Common CLI commands include ls / , create /test , get /test , set /test hello , and delete /test . The configuration file zoo.cfg defines parameters such as tickTime=2000 (heartbeat interval), initLimit=10 (leader‑follower connection limit), syncLimit=5 (sync timeout), dataDir (data storage path), and clientPort=2181 (client connection port).

ZooKeeper maintains a Stat structure for each ZNode, tracking version numbers ( version , cversion , aversion ), timestamps, transaction IDs, data length, and child count. Sessions represent long‑lived TCP connections between clients and servers, identified by a unique SessionID and governed by timeout, tickTime, and closing flags.

Access control is enforced via ACLs with permissions CREATE, READ, WRITE, DELETE, and ADMIN. Cluster roles consist of Leader (handles reads/writes and initiates votes), Follower (serves reads and participates in elections), and Observer (read‑only, does not vote). The leader election process follows a quorum‑based approach, requiring a majority of servers to agree on the new leader.

Watchers are lightweight one‑time triggers that notify clients of data or child changes; they are stored in the client’s WatchManager and re‑registered upon reconnection if necessary. Persistent watches (available from ZooKeeper 3.6) remain after being triggered.

References include the book "From Paxos to ZooKeeper: Distributed Consensus Principles and Practice", Alibaba middleware blog, the official ZooKeeper documentation, and various online tutorials.