Question

A machine learning team runs a real-time SageMaker endpoint for image classification on ml.g4dn.xlarge instances. Monitoring shows GPU utilization averages only 12%, but inference latency is acceptable and traffic is steady throughout the day. Finance has asked the team to reduce the endpoint's compute cost without significantly degrading latency or rewriting the model. Which action best addresses this requirement?

Accepted Answer

Run SageMaker Inference Recommender to benchmark the model across CPU and smaller instance families, then redeploy to the most cost-effective instance type that meets the latency SLA.

Answer

Enable target-tracking auto scaling on the existing GPU endpoint so idle instances are removed automatically.

Answer

Switch the endpoint to Serverless Inference to eliminate all instance charges.

Answer

Increase the instance count to two ml.g4dn.xlarge instances to spread load and improve GPU efficiency.