Technical Overview of Kube-OVN Deployment for Hybrid VM and Container Environments at ByteDance

At KubeCON China 2021, ByteDance senior R&D engineer Fei Xiang presented the team's practical experience with Kube-OVN, describing the technology selection process and the extensions they built on top of the community version for their hybrid VM‑and‑container scenario.

The team chose Kube-OVN because it provides centralized IPAM for flexible address management, a VPC/Subnet model that aligns with traditional IaaS networking, an OVN‑based control plane for advanced orchestration, and an OVS data plane that supports offload and DPDK acceleration.

The first network design implements a pod traffic model where pods communicate directly over Geneve tunnels, a distributed gateway subnet routes external traffic through the node's ovn0 interface, and VM pods use underlay addresses with ECMP high‑availability paths.

The service traffic model uses IPVS, assigns external IPs to publicly exposed services, and publishes external service BGP prefixes.

After deployment, three main problems were observed: an excessive number of source routes per pod affecting performance at scale, overly long traffic paths for VM workloads, and limited industry adoption of Geneve tunnels leading to compatibility risks.

To address these issues, the team introduced three improvements: (1) replace the kernel OVS datapath with OVS‑DPDK and create a vhost‑client socket for VM pods; (2) optimize source routing by creating a public logical switch (172.168.0.1/30) with a localnet port, issuing a default route (0.0.0.0/0 via 172.168.0.2) and configuring MAC‑binding entries; (3) replace Geneve with VXLAN, simplifying tunnel headers and adjusting OVS flow tables (tables 0, 6, 33) to bind MAC addresses and avoid port conflicts.

Looking ahead, the community hopes to add multi‑cluster interconnect, tighter VPC‑style networking, richer service capabilities, full NFV features (LB, NAT, VPN), more comprehensive observability tools, and possibly a data‑plane implementation that does not depend on OVN.

For more information, visit the Kube‑OVN website, GitHub repository, and Slack channel, or join the community WeChat group.