Question

Your team ran 15 separate training jobs in Azure Machine Learning, each logged to the same MLflow experiment with a logged metric named 'val_f1_score'. A stakeholder asks you to identify which single job produced the highest validation F1 score so it can be promoted for registration. You want the fastest approach that requires no re-execution of the jobs. What should you do?

Accepted Answer

Open the experiment in Azure Machine Learning studio, add the jobs to a comparison view, and sort or chart the runs by the logged 'val_f1_score' metric.

Answer

Re-run all 15 jobs as a single hyperparameter sweep job so the best model is automatically selected by the primary metric.

Answer

Download the output artifacts of every job and manually inspect each scoring log file to find the highest value.

Answer

Deploy each of the 15 models to a separate online endpoint and send test traffic to measure which returns the best F1.