Question

An operator manages a shared A100 GPU node that is currently partitioned into seven 1g.5gb MIG instances, all running inference pods. A data science team requests a larger 3g.20gb MIG instance for a memory-heavy model. When the operator attempts to reconfigure the GPU into the new MIG geometry, the operation fails. What is the most likely reason and the correct remediation?

Accepted Answer

MIG geometry cannot be changed while instances are in use; the operator must drain and stop all workloads on that GPU before reconfiguring the MIG profile.

Answer

The A100 does not support the 3g.20gb profile; the operator should switch to vGPU time-slicing instead to satisfy the memory requirement.

Answer

MIG mode must be disabled cluster-wide before any single GPU can be repartitioned; the operator should reboot all nodes.

Answer

The failure is caused by an outdated GPU driver; upgrading DCGM on the node will allow live MIG resizing without stopping pods.