How KubeZoo Enables Lightweight Multi‑Tenant Kubernetes: A Deep Dive

Kubernetes Multi‑Tenant Model Overview

KubeZoo is a lightweight multi‑tenant project developed by ByteDance, built on the core idea of protocol conversion. It virtualizes multiple tenants on a single physical Kubernetes master, providing lightweight, native‑API‑compatible, and non‑intrusive multi‑tenant capabilities, making it an excellent solution for building a serverless Kubernetes foundation.

Why Multi‑Tenant Kubernetes Matters

Since Kubernetes became the de‑facto standard for container orchestration, managing multi‑tenant clusters remains a key challenge for developers and enterprises. In private clouds, many platforms hide Kubernetes details, yet some workloads still require dedicated Kubernetes clusters, leading to high control‑plane costs and inefficient resource usage.

Common Multi‑Tenant Models

Namespace as a Service (NaaS) : Tenants share the same control plane and compute pool, each confined to a namespace. This offers low operational cost but limits tenants to namespace‑scoped resources.

Cluster as a Service (CaaS) : Each tenant receives a full, isolated cluster with its own master and nodes, providing strong isolation at higher resource cost.

Control Planes as a Service (CPaaS) : Tenants have separate masters (virtual clusters) but share the underlying compute pool, balancing isolation and resource efficiency.

Architecture Overview

KubeZoo consists of a

kubezoo‑server

process and an etcd store. The server is stateless and can be deployed with multiple masters for horizontal scaling, while etcd holds lightweight tenant metadata.

The system sits in front of a Kubernetes cluster, receiving user requests, translating them, and forwarding them to the upstream cluster. It then returns the results to the user.

Tenant Management

KubeZoo defines a

Tenant

object with a globally unique six‑character name, stored in etcd. When a tenant is created, KubeZoo issues an X509 certificate containing the tenant name, syncs the tenant’s namespace and RBAC to the upstream cluster, and stores metadata in etcd. Deleting a tenant triggers cleanup of all associated resources.

Security Design

KubeZoo supports both X509 certificates and ServiceAccount tokens for authentication. Both rely on the same CA as the upstream Kubernetes master. For X509, the tenant name is placed in the

Subject.OrganizationalUnit

field; for ServiceAccounts, the tenant information is derived from the namespace prefix.

Traffic Management

Using a token‑bucket algorithm, KubeZoo enforces per‑tenant rate limiting. Administrators can set the maximum concurrent requests per tenant via the annotation

tce.kubezoo/max-requests-inflight

. Requests are processed only if a token is available; otherwise they are rejected.

Summary

KubeZoo provides a lightweight, native‑API‑compatible multi‑tenant solution that balances cost, efficiency, and security. It offers a promising path for building serverless‑ready Kubernetes platforms, with open‑source code available on GitHub.