Question

A data science team runs a computer-vision training pipeline on a multi-GPU server. During training, GPU utilization repeatedly dips to 40-50%, while the host CPUs sit pinned near 100% performing JPEG decode and augmentation for the input pipeline. Storage bandwidth and network are well within limits. What is the most appropriate infrastructure adjustment to improve GPU utilization?

Accepted Answer

Increase the CPU-to-GPU balance by provisioning more CPU cores (or offloading decode/augmentation to the GPU) so the data pipeline can keep the GPUs fed

Answer

Add faster NVMe storage to the server to raise read throughput

Answer

Replace the GPUs with higher-memory GPU models to fit larger batches

Answer

Add an InfiniBand adapter to increase inter-node bandwidth