Question

You administer a Kubernetes cluster serving a development team that runs many small, lightweight inference notebooks. Each notebook uses only a fraction of an A100 GPU's compute and memory, but developers complain that pods stay 'Pending' because each requests one whole GPU. You want to let multiple pods share a single physical GPU while requiring minimal reconfiguration and no guarantee of hardware-level isolation between workloads. Which approach best fits these requirements?

Accepted Answer

Enable GPU time-slicing in the NVIDIA GPU Operator so a single GPU advertises multiple shareable replicas to the scheduler

Answer

Partition each A100 into MIG instances and expose them as separate schedulable resources

Answer

Deploy vGPU with per-VM frame-buffer profiles to divide each GPU among the notebooks

Answer

Add node taints so only inference pods can be scheduled on the GPU nodes