Question

A machine learning engineer is configuring a SageMaker Automatic Model Tuning (hyperparameter tuning) job for a custom training container that trains a text classifier. The container writes evaluation output to stdout, including a line like 'validation_macro_f1: 0.8123' at the end of each epoch. The engineer wants the tuning job to optimize for the highest macro F1 score on the validation set. Which configuration correctly enables the tuning job to track and optimize this metric?

Accepted Answer

Define an objective metric with a MetricName and a regex expression in the tuning job's metric definitions that extracts the numeric value from the stdout line, and set the objective type to Maximize.

Answer

Enable SageMaker Debugger and create a built-in rule that captures the macro F1 value from tensor collections, then reference the rule name as the tuning objective.

Answer

Publish the macro F1 value as a custom CloudWatch metric from the container and point the tuning job's objective metric directly at the CloudWatch namespace.

Answer

Set the objective type to Minimize and use the built-in 'validation:f1' metric name, since SageMaker automatically parses standard classification metrics from any container.