Question

A research team is building a recommender system whose embedding tables are far too large to fit in GPU VRAM, forcing constant data movement between the CPU's large system memory and the GPU. They want a single platform that tightly couples a CPU and GPU with a high-bandwidth, cache-coherent link so the GPU can access the CPU's memory pool efficiently without traditional PCIe bottlenecks. Which NVIDIA platform best fits this requirement?

Accepted Answer

NVIDIA GH200 Grace Hopper Superchip, which connects a Grace CPU and Hopper GPU over NVLink-C2C with coherent memory access

Answer

Two separate A100 PCIe GPUs installed in a standard x86 server connected by PCIe Gen4

Answer

An HGX H100 baseboard with eight GPUs interconnected only by NVSwitch for GPU-to-GPU traffic

Answer

A cluster of L4 inference GPUs connected across nodes using InfiniBand