Question

In a shared Kubernetes cluster, an operations team runs both latency-sensitive inference pods and long-running training jobs on GPU nodes. They want to reserve a dedicated pool of eight H100 nodes exclusively for the inference workload so that training jobs never land there, while inference pods are still guaranteed to schedule onto those nodes. Which combination of Kubernetes mechanisms best accomplishes this?

Accepted Answer

Apply a taint to the eight inference nodes and add a matching toleration plus a node affinity/nodeSelector to the inference pods

Answer

Set a higher pod priority class on the inference pods so the scheduler preempts training jobs cluster-wide

Answer

Increase the GPU resource requests on training jobs so they are forced onto the non-inference nodes

Answer

Enable GPU time-slicing on the eight inference nodes so both workloads can share the same GPUs