Question

A machine learning team currently trains a large model on a single 8-GPU node, and one full training epoch takes roughly 24 hours. Business stakeholders now require the same model to complete an epoch in about 3 hours to accelerate experimentation. Assuming the workload scales well across nodes and the interconnect is not the bottleneck, which infrastructure sizing decision best meets this target?

Accepted Answer

Scale out to approximately 8 nodes (64 GPUs) connected with high-speed InfiniBand for multi-node data-parallel training

Answer

Add more system RAM and faster CPUs to the existing single node while keeping the same 8 GPUs

Answer

Replace the local NVMe scratch drives with a larger object storage tier to hold more training data

Answer

Enable Multi-Instance GPU (MIG) to partition each of the 8 GPUs into smaller instances