Question

A research team runs multi-GPU training on a single DGX system with 8 GPUs. Their framework performs frequent all-reduce operations to synchronize gradients across all GPUs during every training step. They notice that when GPUs must exchange data through the CPU's PCIe root complex, throughput drops significantly. Which characteristic of the NVSwitch fabric in this DGX system most directly improves the performance of these all-reduce collective operations?

Accepted Answer

It provides full non-blocking all-to-all connectivity so every GPU can communicate with every other GPU at full NVLink bandwidth simultaneously

Answer

It increases the clock frequency of each GPU's tensor cores to accelerate gradient computation

Answer

It expands the total HBM memory capacity available to each individual GPU

Answer

It replaces the InfiniBand adapters needed for scaling training across multiple DGX nodes