Question

A data science team runs a large image-classification training job on an 8-GPU server. Profiling shows GPUs frequently idle while waiting for data, and CPU utilization spikes during data loading because every read is copied from the NVMe storage into system memory and then into GPU memory. The storage array itself has plenty of unused bandwidth. Which NVIDIA technology should the team adopt to reduce this data-path overhead?

Accepted Answer

NVIDIA GPUDirect Storage, enabling a direct data path between storage and GPU memory that bypasses the CPU bounce buffer

Answer

NVIDIA NVLink, to increase the bandwidth between the GPUs during gradient exchange

Answer

NVIDIA MIG, to partition each GPU so more instances can share the incoming data stream

Answer

NVIDIA TensorRT, to compile the model for lower-latency inference kernels