Question

A data science team is training a large transformer model and finds that GPU utilization is high but training epochs are taking longer than expected. A senior engineer suggests enabling mixed-precision training to leverage a specialized hardware feature present in the team's modern NVIDIA data center GPUs. Which GPU hardware component is specifically designed to accelerate the matrix-multiply-and-accumulate operations used in mixed-precision deep learning?

Accepted Answer

Tensor Cores, which perform fused matrix multiply-accumulate operations at reduced precision for higher throughput

Answer

CUDA Cores, which execute general-purpose scalar floating-point instructions one thread at a time

Answer

The GPU's L2 cache, which stores frequently accessed model weights closer to the compute units

Answer

The PCIe controller, which manages high-speed data transfer between the CPU and GPU memory