Question

A machine learning team runs long deep-learning training jobs (6-10 hours each) on SageMaker using ml.p3.2xlarge instances. Finance has asked them to reduce training costs significantly, and the jobs are not time-critical as long as they eventually complete. The team is concerned about losing progress if the underlying capacity is reclaimed. Which approach BEST meets these requirements?

Accepted Answer

Enable SageMaker managed spot training and configure checkpointing to an S3 location so interrupted jobs can resume.

Answer

Switch to SageMaker Serverless Inference to avoid paying for idle instance time during training.

Answer

Reduce the instance count to one and disable checkpointing to minimize storage costs.

Answer

Move the training jobs to on-demand ml.p3.8xlarge instances to finish faster and pay for less total runtime.