GPU and HPU Performance Optimization

Overview

The On-Prem AI Infrastructure Platform tunes every layer between the workload and the accelerator. Compute flavors pin CPUs and align memory and PCI devices to the same NUMA node, Kubernetes clusters run the vendor’s own accelerator operator, multi-node training runs over RDMA with GPUDirect, and accelerator telemetry flows into Prometheus and Grafana.

Performance settings apply to both GPU virtual machines and GPU node groups in Kubernetes clusters, because node groups run on the same tuned flavors.

Optimization Layers

Compute
Kubernetes
Multi-Node Training
Monitoring

Polystack Compute flavors for accelerator workloads apply:

Setting	Effect
CPU pinning	Dedicates physical cores to the instance and removes CPU contention
NUMA affinity	Places instance CPUs and memory on the same NUMA node
Hugepages	Backs instance memory with large pages, reducing TLB overhead
PCI NUMA affinity	Attaches the accelerator from the same NUMA node as the instance’s CPUs and memory

Each GPU node group runs its vendor’s operator:

Operator	Accelerators	Capabilities
NVIDIA GPU Operator	NVIDIA GPUs	Driver and runtime management, MIG partitioning, time-slicing, DCGM monitoring
Intel Gaudi Operator	Intel Gaudi HPUs	Gaudi driver, device plugin, and runtime management
AMD GPU Operator	AMD GPUs	ROCm driver, device plugin, and runtime management

Distributed training across nodes uses:

Component	Role
SR-IOV / RDMA networking	High-bandwidth, low-latency interconnect between training nodes
NVIDIA Network Operator	Manages RDMA-capable networking in the cluster and enables GPUDirect
GPUDirect	Moves data directly between GPU memory and the network, bypassing the CPU

Exporter	Accelerators	Destination
DCGM exporter	NVIDIA GPUs	Prometheus and Grafana
ROCm exporter	AMD GPUs	Prometheus and Grafana

Accelerator utilization, memory, temperature, and health metrics appear alongside the rest of the platform in Polystack Monitoring.

GPU Sharing in Kubernetes

MIG (Multi-Instance GPU)

Supported NVIDIA GPUs are partitioned into isolated GPU instances, each with dedicated compute and memory. The NVIDIA GPU Operator manages the MIG configuration on each node.

Time-slicing

Multiple pods share one NVIDIA GPU in turns. Time-slicing suits development and lightweight inference workloads that do not need a full GPU.

Next Steps

Multi-Vendor Accelerator Support

How vendor operators map to node groups and cluster templates.

MLOps Platform

Distributed training with the Kubeflow Training Operator.

Hypervisor Configuration

Hypervisor settings behind accelerator-optimized flavors.

Monitoring

Dashboards and alerting for platform and accelerator metrics.

Core Services

Other Services

Overview

Optimization Layers

Data Path for Multi-Node Training

Next Steps

Multi-Vendor Accelerator Support

MLOps Platform

Hypervisor Configuration

Monitoring

​Overview

​Optimization Layers

​Data Path for Multi-Node Training

​GPU Sharing in Kubernetes

​Next Steps

Multi-Vendor Accelerator Support

MLOps Platform

Hypervisor Configuration

Monitoring

Overview

Optimization Layers

Data Path for Multi-Node Training

GPU Sharing in Kubernetes

Next Steps