Question

A data science team trains a new version of a customer churn classifier weekly. Each version is registered as a new model package version in an existing SageMaker Model Registry model package group. Before promoting a new version to production, the team must compare its offline evaluation metrics (AUC, F1) against the currently approved production version and record the decision, keeping a full history of which version was approved and by whom. Which approach best meets these requirements with the least custom development?

Accepted Answer

Store each version's metrics in the ModelMetrics of the model package, query and compare versions in the Model Registry, and update the ModelApprovalStatus to Approved for the chosen version.

Answer

Write each model's metrics to a custom DynamoDB table and build a Lambda function that compares them and copies the winning model artifact to a fixed 'production' S3 prefix.

Answer

Save evaluation metrics as CloudWatch custom metrics and use a CloudWatch dashboard to visually compare AUC across versions before manually redeploying the endpoint.

Answer

Enable SageMaker Model Monitor on the production endpoint and rely on its drift reports to decide whether the newest trained version should replace the current one.