Question

On a shared Kubernetes GPU cluster, a high-priority production inference deployment cannot schedule because all GPUs are consumed by lower-priority batch training jobs that have no completion deadline. The platform team wants production pods to automatically evict batch pods when GPUs are scarce, without manually deleting workloads. Which Kubernetes mechanism should they configure to achieve this?

Accepted Answer

Define a PriorityClass with a higher value for the inference pods and enable pod preemption so the scheduler evicts lower-priority batch pods

Answer

Add a nodeSelector to the inference deployment so it only lands on GPU nodes

Answer

Increase the resource requests of the batch training pods so they voluntarily release GPUs

Answer

Apply a NoSchedule taint to all GPU nodes and tolerate it only on inference pods