Question

A systems engineer is configuring a PCIe-based server with four GPUs for a data-loading-intensive inference pipeline. During validation, GPUs 2 and 3 show significantly reduced host-to-device transfer bandwidth compared to GPUs 0 and 1, even though all four cards are identical. Reviewing the server topology, the engineer finds that GPUs 2 and 3 are connected through a PCIe switch that shares a single upstream x16 link to the CPU, while GPUs 0 and 1 each have dedicated x16 links. What is the most likely cause of the observed bandwidth difference?

Accepted Answer

GPUs 2 and 3 are contending for the shared upstream x16 link, halving effective host-to-device bandwidth when both transfer simultaneously

Answer

GPUs 2 and 3 have less GPU memory bandwidth because they are behind a PCIe switch

Answer

The PCIe switch converts the connection to NVLink, which is slower for host transfers

Answer

GPUs 2 and 3 are running at a lower GPU clock speed due to thermal throttling from switch heat