Question

An ML engineer runs a fraud-detection endpoint monitored by two SageMaker Model Monitor jobs: one for data quality drift and one for model quality (accuracy) degradation. Each job publishes a CloudWatch metric, and separate CloudWatch alarms already trigger an automated retraining pipeline via EventBridge. The team complains that during noisy input spikes, the data-quality alarm fires transient alerts and needlessly launches expensive retraining jobs, even when model accuracy is still acceptable. They want retraining to be triggered only when BOTH drift is detected AND accuracy has actually degraded, without deleting the existing individual alarms used for dashboards. What is the most operationally efficient way to meet this requirement?

Accepted Answer

Create a CloudWatch composite alarm with an ALARM rule that requires both the data-quality drift alarm and the model-quality alarm to be in ALARM state, and route only the composite alarm to the retraining EventBridge rule.

Answer

Increase the evaluation periods and datapoints-to-alarm on the data-quality alarm so it only fires after prolonged drift, and keep it as the sole retraining trigger.

Answer

Delete the two individual alarms and create a single metric-math alarm that averages the drift and accuracy metrics into one combined threshold.

Answer

Add a Lambda function subscribed to both alarm SNS topics that maintains state in DynamoDB and starts retraining only after receiving both notifications within a time window.