Source Code Analysis of kube-proxy IPVS Mode in Kubernetes

kube-proxy currently supports three load‑balancing implementations: userspace, iptables, and IPVS. The first two encounter performance bottlenecks as the number of Services grows, making IPVS the preferred production mode. This article analyses the IPVS implementation in depth.

Command‑line initialization with cobra

func main() {
  command := &cobra.Command{
    Use:   "echo [string to echo]",
    Short: "Echo anything to the screen",
    Long: `echo is for echoing anything back.Echo works a lot like print, except it has a child command.`,
    Args: cobra.MinimumNArgs(1),
    Run: func(cmd *cobra.Command, args []string) {
      fmt.Println("Print: " + strings.Join(args, " "))
    },
  }
  command.Execute()
}

The Run method of the cobra command is the entry point for kube‑proxy's core logic.

ProxyServer.Run implementation

// Run runs the specified ProxyServer.
func (o *Options) Run() error {
  defer close(o.errCh)
  proxyServer, err := NewProxyServer(o)
  if err != nil {
    return err
  }
  if o.CleanupAndExit {
    return proxyServer.CleanupAndExit()
  }
  o.proxyServer = proxyServer
  return o.runLoop()
}

Key steps performed by Run include initializing a ProxyServer instance, optionally cleaning up existing rules, and entering the main run loop.

ProxyServer structure

type ProxyServer struct {
  Client                 clientset.Interface
  EventClient            v1core.EventsGetter
  IptInterface           utiliptables.Interface
  IpvsInterface          utilipvs.Interface
  IpsetInterface         utilipset.Interface
  execer                 exec.Interface
  Proxier                proxy.ProxyProvider
  Broadcaster            record.EventBroadcaster
  Recorder               record.EventRecorder
  ConntrackConfiguration kubeproxyconfig.KubeProxyConntrackConfiguration
  Conntracker            Conntracker // if nil, ignored
  ProxyMode              string
  NodeRef                *v1.ObjectReference
  CleanupIPVS            bool
  MetricsBindAddress     string
  EnableProfiling        bool
  OOMScoreAdj            *int32
  ConfigSyncPeriod       time.Duration
  HealthzServer          *healthcheck.HealthzServer
}

The struct holds clients for the API server, interfaces for iptables, IPVS, ipset, and the selected Proxier based on the ProxyMode flag.

Proxier structure (IPVS mode)

type Proxier struct {
  endpointsChanges *proxy.EndpointChangeTracker
  serviceChanges   *proxy.ServiceChangeTracker
  // ...
  serviceMap   proxy.ServiceMap
  endpointsMap proxy.EndpointsMap
  portsMap     map[utilproxy.LocalPort]utilproxy.Closeable
  // ...
  iptables     utiliptables.Interface
  ipvs         utilipvs.Interface
  ipset        utilipset.Interface
  exec         utilexec.Interface
  // ...
  ipvsScheduler string
}

The Proxier maintains maps of services and endpoints and uses the async.BoundedFrequencyRunner to periodically invoke syncProxyRules .

BoundedFrequencyRunner definition

type BoundedFrequencyRunner struct {
  name        string // the name of this instance
  minInterval time.Duration // the min time between runs, modulo bursts
  maxInterval time.Duration // the max time between runs
  run         chan struct{} // try an async run
  mu          sync.Mutex // guards runs of fn and all mutations
  fn          func() // function to run
  lastRun     time.Time // time of last run
  timer       timer // timer for deferred runs
  limiter     rateLimiter // rate limiter for on-demand runs
}

It is created in NewProxier as:

proxier.syncRunner = async.NewBoundedFrequencyRunner(
    "sync-runner", proxier.syncProxyRules, minSyncPeriod, syncPeriod, burstSyncs)

ServiceConfig and ServiceHandler

type ServiceConfig struct {
  listerSynced  cache.InformerSynced
  eventHandlers []ServiceHandler
}

type ServiceHandler interface {
  OnServiceAdd(service *v1.Service)
  OnServiceUpdate(oldService, service *v1.Service)
  OnServiceDelete(service *v1.Service)
  OnServiceSynced()
}

ServiceConfig registers event handlers with the Kubernetes informer to watch Service objects. Example creation:

func NewServiceConfig(serviceInformer coreinformers.ServiceInformer, resyncPeriod time.Duration) *ServiceConfig {
  result := &ServiceConfig{listerSynced: serviceInformer.Informer().HasSynced}
  serviceInformer.Informer().AddEventHandlerWithResyncPeriod(
    cache.ResourceEventHandlerFuncs{
      AddFunc:    result.handleAddService,
      UpdateFunc: result.handleUpdateService,
      DeleteFunc: result.handleDeleteService,
    },
    resyncPeriod,
  )
  return result
}

Handlers forward events to the Proxier, e.g.:

func (c *ServiceConfig) handleAddService(obj interface{}) {
  service, ok := obj.(*v1.Service)
  if !ok { utilruntime.HandleError(fmt.Errorf("unexpected object type: %v", obj)); return }
  for i := range c.eventHandlers { c.eventHandlers[i].OnServiceAdd(service) }
}

Proxier implements these methods to update its internal serviceChanges and trigger a sync run.

func (proxier *Proxier) OnServiceAdd(service *v1.Service) { proxier.OnServiceUpdate(nil, service) }
func (proxier *Proxier) OnServiceUpdate(oldService, service *v1.Service) {
  if proxier.serviceChanges.Update(oldService, service) && proxier.isInitialized() {
    proxier.syncRunner.Run()
  }
}

Core rule synchronization – syncProxyRules

func (proxier *Proxier) syncProxyRules() {
  // Build IPVS rules for each service.
  for svcName, svc := range proxier.serviceMap {
    // ... handle endpoints, ipset entries, virtual server creation ...
    serv := &utilipvs.VirtualServer{Address: svcInfo.ClusterIP(), Port: uint16(svcInfo.Port()), Protocol: string(svcInfo.Protocol()), Scheduler: proxier.ipvsScheduler}
    if svcInfo.SessionAffinityType() == v1.ServiceAffinityClientIP {
      serv.Flags |= utilipvs.FlagPersistent
      serv.Timeout = uint32(svcInfo.StickyMaxAgeSeconds())
    }
    if err := proxier.syncService(svcNameString, serv, true); err == nil {
      // sync endpoints, ipsets, iptables, etc.
    } else {
      klog.Errorf("Failed to sync service: %v, err: %v", serv, err)
    }
  }
  // sync ipset entries and write iptables rules
  for _, set := range proxier.ipsetList { set.syncIPSetEntries() }
  proxier.writeIptablesRules()
  proxier.iptablesData.Reset()
  proxier.iptablesData.Write(proxier.natChains.Bytes())
  proxier.iptablesData.Write(proxier.natRules.Bytes())
  proxier.iptablesData.Write(proxier.filterChains.Bytes())
  proxier.iptablesData.Write(proxier.filterRules.Bytes())
}

The function iterates over the service map, creates IPVS virtual servers, handles SNAT, external IPs, load‑balancer ingress, NodePort, updates ipset entries, and finally writes iptables rules.

Summary

The kube‑proxy code follows a clear design: it watches Service and Endpoint resources, records changes in ServiceMap and EndpointsMap , and uses an asynchronous BoundedFrequencyRunner to invoke syncProxyRules , which translates the desired state into IPVS, ipset, and iptables configurations.