Question

A company hosts several real-time SageMaker endpoints for internal analytics teams. The ML platform lead notices the monthly inference bill has grown steadily, but many endpoints show very low invocation counts during business hours and none overnight. The lead wants an automated, low-effort way to identify underutilized endpoints so that the team can decommission or consolidate them, while avoiding manual review of each endpoint. Which approach best meets this goal?

Accepted Answer

Publish the SageMaker Invocations, CPUUtilization, and MemoryUtilization metrics to a CloudWatch dashboard and set CloudWatch alarms that notify the team via SNS when Invocations remains near zero over a multi-day period.

Answer

Create a CloudWatch alarm on the InvocationsPerInstance metric with a low static threshold and configure it to terminate the endpoint automatically.

Answer

Enable SageMaker Model Monitor data-quality monitoring on each endpoint to detect when no data is being captured.

Answer

Switch all endpoints to SageMaker Batch Transform so that idle compute is never provisioned.