Question

A data center team is deploying a single DGX H100 server to train a large model where every one of the 8 GPUs must exchange gradient data with every other GPU at full bandwidth simultaneously. During procurement review, a colleague suggests that point-to-point NVLink cables between GPU pairs are sufficient and that no additional switching component is needed. Why does the DGX H100 architecture include NVSwitch rather than relying only on direct NVLink links between GPU pairs?

Accepted Answer

NVSwitch provides a non-blocking switching fabric that lets all 8 GPUs communicate with each other at full NVLink bandwidth simultaneously, rather than being limited by a fixed number of direct pairwise links

Answer

NVSwitch replaces the need for PCIe entirely, connecting the GPUs directly to system storage for faster checkpoint writes

Answer

NVSwitch converts NVLink traffic into InfiniBand packets so the GPUs can communicate across multiple nodes without extra NICs

Answer

NVSwitch increases the clock speed of each GPU's NVLink ports to compensate for the lack of direct cabling between GPU pairs