Running Kubernetes Across Multiple Zones: Design Principles and Operational Practices

Background – Kubernetes is designed so a single cluster can span multiple failure zones, which are grouped into logical regions. Cloud providers define a region as a set of failure zones (availability zones) that provide consistent APIs and services, minimizing the risk that a failure in one zone impacts another.

Control Plane Behavior – All control‑plane components run as a pool of interchangeable resources, each replicated. For high availability, deploy control‑plane replicas across at least three failure zones, including API server, scheduler, etcd, and controller manager, and also replicate the cloud‑controller manager if used. The API server itself lacks built‑in cross‑region elasticity, so external techniques like DNS round‑robin, SRV records, or third‑party load balancers with health checks are recommended.

Node Behavior – Kubernetes automatically spreads pods of workloads (Deployments, StatefulSets, etc.) across nodes to reduce impact of failures. When a node starts, kubelet adds labels to the node object, which can include zone information. These labels can be combined with pod topology spread constraints to control how pods are placed across zones, regions, or specific nodes, improving expected availability.

Cross‑Zone Node Distribution – Nodes are not created by Kubernetes itself; you must provision them manually or via tools like Cluster API. Such tools let you define a set of worker machines that run across multiple failure zones and automatically repair the cluster when an entire zone goes down.

Manual Zone Assignment for Pods – You can apply node‑selector constraints to Pods or to the pod templates of higher‑level workloads (Deployments, StatefulSets, Jobs) to enforce placement in specific zones.

Zone‑Aware Storage Access – When a PersistentVolume is created, the PersistentVolumeLabel controller adds a zone label to it. The scheduler’s NoVolumeZoneConflict predicate ensures that a pod requesting that volume is scheduled only on nodes in the same zone. You can also specify a StorageClass that limits volumes to particular failure zones.

Network – Kubernetes itself is not zone‑aware for networking; you must use a CNI plugin that supports zone‑specific features. If the cloud provider offers a LoadBalancer service type, traffic may be confined to the same zone as the load‑balancer endpoint. Custom or on‑prem deployments need to consider similar zone‑related networking behavior.

Fault Recovery – When planning a cluster, consider scenarios where all failure zones in a region become unavailable. Ensure that critical repair jobs do not depend on the existence of at least one healthy node, and design recovery procedures that can operate even when no nodes are initially functional.