Unifying GPU Management Across Kubernetes Clusters with RBAC & Virtual Control Planes

Background

To centrally manage GPU resources at the group level, the platform needs to present and manage container public clusters, search clusters, and HBox clusters uniformly. The challenge is sharing GPU compute power across departmental clusters.

Current Situation

HBox holds most GPU compute and will become the centralized GPU management department.

Search has admin rights to HBox and can schedule GPU pods on authorized nodes.

The commercial department hosts GPU machines in the public cluster with high utilization and wants a K8s container platform for GPU instance management.

Analysis

A capability is required to interconnect resources across departments and clusters with permission control.

The solution must not disrupt existing GPU usage patterns and should follow mainstream industry directions.

Solution Overview

Common multi‑tenant approaches in the industry include:

Cluster‑internal Namespace & RBAC – isolate each user with a dedicated namespace and role.

Virtual Control Plane – each user gets an independent virtual apiserver for resource control.

Multi‑Cluster with Virtual Control Plane – manage multiple physical clusters using virtual control planes.

K8s Namespace & RBAC Isolation

Principle

RBAC (K8s native) controls access by defining Roles, RoleBindings, ClusterRoles, and ClusterRoleBindings. Roles grant permissions on resources within a namespace, while ClusterRoles apply cluster‑wide.

Role : defines access to resources inside a specific namespace.

RoleBinding : binds users or groups to a Role.

ClusterRole : defines access to resources across the whole cluster.

ClusterRoleBinding : binds users or groups to a ClusterRole.

Example ClusterRole that allows read‑only access to nodes:

<code>apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: view-nodes-role
rules:
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get","list"]
</code>

Node selection via labels can further restrict permissions, e.g., a ClusterRole for GPU nodes.

Virtual Control Plane (vcluster)

Principle

vcluster creates a virtual Kubernetes cluster inside an existing physical cluster, providing an independent API server, namespace, and RBAC policies for each virtual cluster.

Key components:

Resource isolation – each virtual cluster receives a dedicated quota of CPU, memory, storage.

API server proxy – forwards requests from the virtual API server to the physical one.

Namespace & RBAC management – independent within each virtual cluster.

vcluster syncer – mirrors resources (pods, services, deployments, configmaps, secrets) between virtual and physical clusters, handling bidirectional synchronization, event processing, and consistency.

Security – synchronizes RBAC and security policies.

Typical syncer configuration synchronizes objects and transforms them as needed.

Multi‑Cluster Management Solutions

Karmada

Provides a unified view and scheduling across clouds and data centers, supporting edge‑cloud collaboration and high‑availability disaster recovery. Its focus is on application management; resource‑level granularity is weaker.

Virtual Kubelet

Imitates a Kubernetes node to integrate external resource platforms. It offers non‑intrusive integration, extensible providers, serverless billing, and simplifies multi‑cluster resource pooling, but has scheduling and service‑discovery limitations.

Conclusion

Depending on enterprise needs, choose the appropriate approach: use K8s Namespace & RBAC for fine‑grained permission control within existing clusters, adopt a virtual control plane for stronger isolation and flexibility, or employ multi‑cluster solutions like Karmada or Virtual Kubelet for cross‑cluster scheduling.