Key New Features in Kubernetes v1.29: CEL‑based CRD Validation, NodePort Allocation, Sidecar Containers, PreStop Hook, Service Account Token Binding, and More

Hello, I am Zhang Jintao.

Kubernetes v1.29 is the third major release of 2023 and the last one of the year, bringing 49 major updates. For comparison, v1.27 had nearly 60 updates and v1.28 had 46. Even after almost ten years, Kubernetes remains vibrant.

In this release, 19 enhancements are moving to Alpha, another 19 to Beta, and 11 to Stable, showing a steady flow of new features.

KEP‑2876: CEL‑based CRD Validation Reaches GA

This feature is crucial for anyone extending Kubernetes via CustomResourceDefinitions (CRDs). Native CRD validation currently supports two kinds of checks: structural validation defined by the CRD schema and OpenAPIv3 validation rules.

Structural validation based on the CRD definition

OpenAPIv3 validation rules

Example CRD definition (truncated):

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.13.0
  name: kongplugins.configuration.konghq.com
spec:
  group: configuration.konghq.com
  scope: Namespaced
  versions:
    - name: v1
      schema:
        openAPIV3Schema:
          description: KongPlugin is the Schema for the kongplugins API.
          properties:
            apiVersion:
              description: 'APIVersion defines the versioned schema of this representation of an object. ...'
              type: string
            protocols:
              description: Protocols configures plugin to run on requests received on specific protocols.
              items:
                description: KongProtocol is a valid Kong protocol.
                enum:
                - http
                - https
                - grpc
                - grpcs
                - tcp
                - tls
                - udp
                type: string
              type: array
          type: object
          x-kubernetes-validations:
          - message: Using both config and configFrom fields is not allowed.
            rule: '!(has(self.config) && has(self.configFrom))'
          - message: Using both configFrom and configPatches fields is not allowed.
            rule: '!(has(self.configFrom) && has(self.configPatches))'
          - message: The plugin field is immutable
            rule: self.plugin == oldSelf.plugin

The example defines a custom resource KongPlugin that uses openAPIV3Schema for validation. While native rules are limited, richer validation can be achieved with Admission Webhooks or custom validators such as OPA Gatekeeper, but these add complexity.

To simplify, the community introduced CEL‑based validation that can be written directly in the CRD using x-kubernetes-validations , eliminating the need for external webhooks or validators.

KEP‑3668: Stable Static and Dynamic NodePort Allocation Range

Kubernetes Service objects of type NodePort expose services outside the cluster. Previously, manually assigning a specific NodePort could lead to conflicts.

KEP‑3668 reserves a static sub‑range within the overall NodePort range (default 30000‑32767) and allows the rest to be allocated dynamically. The calculation uses the formula min(max($min, node-range-size/$step), $max) . For the default range:

Service NodePort range: 30000‑32767

Range size: 2767

Band offset: min(max(16,2767/32),128) = 86

Static band start: 30000

Static band end: 30086

Thus ports 30000‑30086 are static; ports above 30086 are dynamic.

┌─────────────┬─────────────────────────────────────────────┐
 │   static   │                    dynamic               │
 └─────────────┴─────────────────────────────────────────────┘

 ◄────────────► ◄────────────────────────────────────────────►
30000        30086                                          32767

When manually assigning a NodePort, choosing a port in the static band reduces the chance of conflict.

KEP‑753: SidecarContainers Reach Beta and Are Enabled by Default

Sidecar containers add auxiliary functionality to a pod (e.g., logging, monitoring). Previously, sidecars could outlive the main container, causing reliability issues.

KEP‑753 moves sidecars into the initContainers list with an "Always" restart policy, ensuring they follow the main container lifecycle. Example:

apiVersion: v1
kind: Pod
metadata:
  name: moelove-pod
spec:
  initContainers:
  - name: log
    image: moelove/fluentbit
    restartPolicy: Always
    ...

In v1.29 this feature is enabled by default, and sidecars will be terminated in reverse order of their start, guaranteeing the main container stops first.

KEP‑3960: PreStop Hook Adds Native Sleep Action (Alpha)

Many applications need a brief pause before termination to gracefully close connections. Previously only exec and httpGet were supported.

KEP‑3960 introduces a native sleep action:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  template:
    spec:
      containers:
      - name: nginx
        image: nginx:1.25.3
        lifecycle:
          preStop:
            sleep:
              seconds: 5
        readinessProbe:
          httpGet:
            path: /
            port: 80

Before this KEP, achieving the same required an exec of sh -c "sleep 5" , which depended on the container having the sleep binary.

KEP‑4193: Improved ServiceAccount Token Binding (Alpha)

ServiceAccount tokens are essential for workload authentication. In v1.29 each ServiceAccount can be bound to a specific Pod instance, preventing token reuse after leakage.

Feature gates controlling this behavior include:

LegacyServiceAccountTokenCleanUp=true|false  (BETA - default=true)
ServiceAccountTokenJTI=true|false          (ALPHA - default=false)
ServiceAccountTokenNodeBinding=true|false  (ALPHA - default=false)
ServiceAccountTokenNodeBindingValidation=true|false (ALPHA - default=false)
ServiceAccountTokenPodNodeInfo=true|false (ALPHA - default=false)

KEP‑727: Kubelet Resource Metrics Reach GA

This long‑standing KEP finally graduates to GA, exposing metrics such as:

container_cpu_usage_seconds_total

container_memory_working_set_bytes

container_start_time_seconds

node_cpu_usage_seconds_total

node_memory_working_set_bytes

pod_cpu_usage_seconds_total

pod_memory_working_set_bytes

resource_scrape_error

Sample output:

# HELP container_cpu_usage_seconds_total [STABLE] Cumulative cpu time consumed by the container in core‑seconds
# TYPE container_cpu_usage_seconds_total counter
container_cpu_usage_seconds_total{container="coredns",namespace="kube-system",pod="coredns-55968cc89d-bhhbx"} 0.195744 1691361886865
# HELP container_memory_working_set_bytes [STABLE] Current working set of the container in bytes
# TYPE container_memory_working_set_bytes gauge
container_memory_working_set_bytes{container="coredns",namespace="kube-system",pod="coredns-55968cc89d-bhhbx"} 1.675264e+07 1691361886865
... (additional metric lines omitted for brevity)

KEP‑3466: Kubernetes Component Health SLIs Reach GA

This KEP makes each component expose its health status as Service Level Indicators, enabling cluster‑wide SLO calculations.

Historically ComponentStatus existed but was deprecated in v1.19. The new health metrics look like:

# HELP kubernetes_healthcheck [STABLE] This metric records the result of a single healthcheck.
# TYPE kubernetes_healthcheck gauge
kubernetes_healthcheck{name="etcd",type="healthz"} 1
kubernetes_healthcheck{name="etcd",type="livez"} 1
kubernetes_healthcheck{name="etcd",type="readyz"} 1
... (additional component lines omitted)

Known Issues

The EventedPLEG feature, promoted to Beta in v1.27, has been disabled by default in v1.29 due to numerous problems discovered during testing.

Other GA Features

KEP‑2495: PV/PVC ReadWriteOncePod reaches GA

NodeExpandSecret reaches GA

kube‑proxy gains a new nftables backend

These are the main highlights of Kubernetes v1.29. See you next time!