Question

A Kubernetes cluster has four nodes, each with one NVIDIA GPU. A team submits eight lightweight inference pods, each requesting 'nvidia.com/gpu: 1'. Four pods schedule and run, but the other four remain in 'Pending' state indefinitely even though the running pods use less than 20% of GPU compute and memory. Cluster admins want all eight pods to run concurrently without buying more hardware. What is the most appropriate action?

Accepted Answer

Configure GPU time-slicing in the NVIDIA GPU Operator so each physical GPU advertises multiple shared replicas of the nvidia.com/gpu resource

Answer

Increase each pod's GPU resource request to 'nvidia.com/gpu: 2' so the scheduler packs them more efficiently

Answer

Add a node anti-affinity rule so the pending pods are forced onto the existing nodes

Answer

Remove the GPU resource request entirely so the pods schedule on any available CPU node