Question

A data center operations team runs several small inference services, each of which only uses a fraction of a single NVIDIA A100 GPU's compute and memory. They want to run multiple isolated workloads on one physical GPU simultaneously, with guaranteed, partitioned resources and fault isolation between tenants. Which NVIDIA GPU capability should they use?

Accepted Answer

Multi-Instance GPU (MIG), which partitions a single GPU into isolated instances with dedicated compute and memory

Answer

NVLink, which increases bandwidth for combining multiple GPUs into one large device

Answer

Tensor Cores, which accelerate mixed-precision matrix math within a workload

Answer

Unified Memory, which lets the GPU and CPU share a single address space