Question

A research lab is specifying a single server to train large transformer models that require frequent all-to-all communication among 8 GPUs within the same chassis. They want the highest possible intra-server GPU-to-GPU bandwidth so that gradient synchronization does not become the bottleneck. Which server building block should they select?

Accepted Answer

An NVIDIA HGX baseboard with 8 SXM GPUs connected via NVSwitch

Answer

A standard 2U server with 8 PCIe GPUs communicating over the PCIe bus

Answer

Eight separate 1U servers each with a single PCIe GPU networked over 25GbE Ethernet

Answer

A CPU-only server with high core count and large system memory