Question

A data science team is running a SageMaker Automatic Model Tuning job for an XGBoost regression model that predicts monthly customer spend. The business cares most about not producing wildly inaccurate predictions for a small number of high-spend customers, since those errors are extremely costly. The team must configure the hyperparameter tuning job's objective metric so tuning selects the model that best serves this business goal. Which objective metric configuration should they choose?

Accepted Answer

Minimize validation:rmse so larger prediction errors are penalized more heavily during model selection

Answer

Maximize validation:r2 to select the model that explains the most variance in customer spend

Answer

Minimize validation:mae so all prediction errors contribute equally regardless of magnitude

Answer

Maximize validation:accuracy to count predictions that exactly match actual spend values