Question

A data science team runs a large image-classification training job on an 8-GPU DGX system. During training they observe that GPU utilization repeatedly drops to 20-30% and the job runs far slower than expected, even though the model and batch size fit comfortably in GPU memory. Monitoring shows the CPUs and GPUs are frequently idle while waiting. What is the most likely infrastructure cause?

Accepted Answer

The storage and data pipeline cannot deliver training samples fast enough to keep the GPUs fed

Answer

The GPUs lack sufficient Tensor Cores for the model's precision requirements

Answer

The NVLink interconnect between GPUs has insufficient bandwidth for gradient exchange

Answer

The model is too large to fit in GPU memory and is being swapped to system RAM