Question

You administer a Kubernetes cluster with the NVIDIA GPU Operator. Some A100 nodes are configured with MIG enabled to serve many small inference pods, while other A100 nodes remain in full-GPU (non-MIG) mode for large training jobs. Data scientists report that inference pods requesting a MIG slice occasionally land on the full-GPU nodes and fail, while training pods sometimes get scheduled onto MIG-enabled nodes. What is the most appropriate way to configure the GPU Operator's MIG strategy so both workload types schedule correctly across the mixed fleet?

Accepted Answer

Set the MIG strategy to 'mixed' so the operator advertises distinct MIG-slice resource names on MIG nodes and full nvidia.com/gpu resources on non-MIG nodes, allowing pods to request the exact resource they need.

Answer

Set the MIG strategy to 'single' on all nodes so every GPU is advertised as one uniform nvidia.com/gpu resource regardless of MIG configuration.

Answer

Disable MIG entirely and rely on Kubernetes time-slicing to share the full GPUs between inference and training pods.

Answer

Set the MIG strategy to 'none' and manually taint every MIG-enabled node so no pods can schedule on them.