Question

A data center team is specifying a new GPU server for training computer-vision models. The GPUs each have 80 GB of HBM, and the workload streams large batches of augmented image data. During a pilot, engineers notice the data-loading and augmentation stage frequently stalls because the OS cannot cache enough of the working dataset and swapping occurs. The GPUs remain underutilized. Which infrastructure adjustment most directly addresses this specific bottleneck?

Accepted Answer

Increase the amount of system (host) RAM so the data pipeline and OS page cache can hold more of the working dataset

Answer

Add more GPUs to the server to raise aggregate HBM capacity

Answer

Replace the InfiniBand adapters with higher-bandwidth NVLink bridges

Answer

Switch the training precision from FP32 to FP16 to reduce GPU memory pressure