Kubeflow Overview: CNCF‑Incubated MLOps Platform on Kubernetes

CNCF Technical Oversight Committee (TOC) vote acceptance: Kubeflow is accepted as a CNCF incubated project.

Kubeflow is an open‑source, community‑driven project for deploying and managing the machine‑learning (ML) stack on Kubernetes. The community actively develops Kubernetes‑based MLOps and supports popular frameworks such as TensorFlow, PyTorch, XGBoost, and Apache MXNet.

Created by Google in 2017, today there are ten commercial distributions of Kubeflow. The project has hundreds of contributors, thousands of users, and over 150 companies participating. Since its launch, Kubeflow has released 15 major versions and provides extensive support to its user base.

The project is tightly integrated with CNCF and the broader ML community. To strengthen its Kubernetes foundation, Kubeflow incorporates Kustomize, Knative, Istio, Certificate Manager, and Argo, simplifying installation, scalability, service mesh, security, and workflow management. It also integrates with gRPC, Prometheus, and is working on integrations with KubeRay and MLflow.

Main Components

Notebooks Working Group: Builds interactive development environments in Jupyter, VS Code, and R‑Studio, and develops the central dashboard and web UI for data visualization.

Training Operator Working Group: Provides the Training Operator for distributed ML training on Kubernetes, supporting multi‑GPU deep‑neural‑network training, various scheduling techniques (e.g., Volcano), elastic training, and SDKs for data scientists.

AutoML Working Group: Develops Katib, an automated model‑development tool offering hyper‑parameter tuning, neural‑architecture search, early‑stopping techniques, and experiment‑tracking UI/SDK.

Kubeflow Pipelines Working Group: Converts Python ML scripts into reusable workflow templates, enabling repeatable, manageable, and observable distributed workflow automation.

Manifests Working Group: Handles the installation process of all Kubeflow components using Kustomize.

KServe Project: Provides a highly scalable, standards‑based model‑inference platform on Kubernetes, now part of Kubeflow’s installation and testing.

The project can be deployed as independent components or as a complete end‑to‑end system.

Notable Milestones

GitHub stars: 28K+

Company contributions: 150+

Total contributors: 15K+

Total GitHub contributions: 55K+

Slack members: 9,000+

15 releases since 2017

Looking ahead, Kubeflow is focusing on its v1.8 roadmap, slated for release in October 2023. New features include Kubeflow Pipelines 2.0 GA, enhanced AutoML experiments, increased scaling capabilities, and Training Operator improvements such as advanced model parallelism and custom scheduler support. Version 1.8 will be tested against specific versions of Kubernetes, Kustomize, Istio, Certificate Manager, Argo, and Knative.

As a CNCF‑hosted project, Kubeflow benefits from neutral governance, marketing support, and community expansion provided by the Linux Foundation. It joins 38 other incubated projects such as Backstage, Cilium, Istio, Knative, and OpenTelemetry. For more details on graduation criteria, see the CNCF graduation standards.

References

CNCF Technical Oversight Committee: https://www.cncf.io/people/technical-oversight-committee/

Kubeflow: https://www.kubeflow.org/

v1.8 Roadmap: https://github.com/orgs/kubeflow/projects/58

Incubation: https://www.cncf.io/projects/

CNCF Graduation Criteria: https://github.com/cncf/toc/blob/main/process/graduation_criteria.md