Question

You are training a large deep learning model on Azure Machine Learning using a compute cluster of 8 GPU nodes. The model no longer fits into the memory of a single GPU, so you need to shard the model parameters, gradients, and optimizer states across all available GPUs to reduce per-device memory consumption. Which approach should you configure for the training job?

Accepted Answer

Use a model-parallel/sharded approach such as DeepSpeed ZeRO to partition parameters, gradients, and optimizer states across GPUs.

Answer

Use data-parallel training where each GPU holds a full copy of the model and processes a different mini-batch.

Answer

Reduce the batch size to 1 and enable gradient accumulation on a single GPU node.

Answer

Submit the job as a parameter sweep so each node trains an independent copy with different hyperparameters.