Question

A data science team is evaluating why their newly acquired NVIDIA GPU dramatically outperforms their old CPU cluster on a deep learning training job. During a design review, a colleague asks specifically what architectural characteristic of the GPU is most responsible for accelerating the dense matrix multiplications at the core of neural network training. Which explanation is most accurate?

Accepted Answer

The GPU contains thousands of relatively simple cores that execute many arithmetic operations concurrently, exploiting the high degree of data parallelism in matrix math.

Answer

The GPU runs at a much higher clock frequency than any CPU, so each individual instruction completes far faster.

Answer

The GPU offloads work to the operating system scheduler, which spreads threads more efficiently than a CPU can.

Answer

The GPU uses a larger last-level cache than the CPU, eliminating the need to access memory during training.