Question

A shared Slurm cluster has several A100 nodes. The platform team wants to give a group of inference users access to small, isolated GPU slices while training users continue to request whole GPUs on other nodes. They have enabled MIG on the inference nodes with 1g.10gb profiles. What must the administrator configure so Slurm can schedule jobs onto the individual MIG instances?

Accepted Answer

Define the MIG instances as distinct GRES/GPU entries in gres.conf and expose corresponding node GRES so jobs can request individual MIG devices

Answer

Enable GPU time-slicing in the Slurm cgroup plugin so each whole GPU is oversubscribed across inference jobs

Answer

Set OverSubscribe=FORCE on the inference partition so multiple jobs share each physical GPU

Answer

Disable MIG and rely on the NVIDIA GPU Operator to partition the devices at pod admission time