Question

A retail company deploys a fraud-detection model to a real-time SageMaker endpoint. Over several weeks, upstream systems change how they populate certain input fields, causing the distribution of incoming feature values to differ from the training data. The ML team wants an automated way to detect when live inference data statistically diverges from the data used to train the model, and to be alerted so they can investigate. Which approach best meets this requirement with the least custom development?

Accepted Answer

Enable SageMaker Model Monitor with a data quality monitoring schedule that compares captured inference data against a baseline computed from the training dataset, and trigger a CloudWatch alarm on the emitted violation metrics.

Answer

Configure SageMaker Model Monitor model quality monitoring against ground-truth labels and alarm when accuracy drops below a threshold.

Answer

Write a scheduled Lambda function that queries the endpoint's CloudWatch invocation and latency metrics and sends an SNS notification when they exceed a threshold.

Answer

Enable SageMaker Debugger with the built-in overfitting and vanishing gradient rules to detect distribution changes during inference.