Question

A data scientist is training a large deep learning model on a tabular dataset of 500 GB using SageMaker. A single ml.p4d.24xlarge instance with 8 GPUs is available, but each epoch takes far too long because the model only uses one GPU. The model fits in a single GPU's memory, and the team wants to reduce training time by distributing the workload across all 8 GPUs on the instance. Which approach should they use?

Accepted Answer

Enable SageMaker's data parallel library to shard each mini-batch across all 8 GPUs and synchronize gradients.

Answer

Enable SageMaker's model parallel library to split the model layers across all 8 GPUs.

Answer

Increase the learning rate proportionally so the single GPU converges in fewer epochs.

Answer

Use SageMaker automatic model tuning to run 8 parallel hyperparameter trials, one per GPU.