Question

A retail company runs a real-time SageMaker endpoint for a credit-approval model. The team wants to automatically detect when the model's predictive accuracy degrades in production and trigger a retraining pipeline. Actual loan repayment outcomes (ground truth labels) become available roughly 30 days after each prediction and are stored in S3. Which approach best enables ongoing detection of accuracy degradation?

Accepted Answer

Enable SageMaker Model Monitor's model quality monitoring job, which merges captured inference data with the ingested ground truth labels to compute metrics like accuracy and F1, and emits CloudWatch metrics that trigger an alarm to start the retraining pipeline.

Answer

Enable SageMaker Model Monitor's data quality monitoring job against a baseline of the training features so that any statistical drift in input distributions automatically triggers retraining.

Answer

Configure SageMaker Clarify bias monitoring to continuously compute pre-training bias metrics on the live endpoint and raise a CloudWatch alarm when bias exceeds a threshold.

Answer

Schedule a nightly Batch Transform job on recent inference inputs and compare its output distribution to the previous night's outputs, alarming when the distributions diverge.