Question

A data scientist enables MLflow autologging for scikit-learn and trains a RandomForestClassifier inside a `with mlflow.start_run()` block. After the model fits, they compute a custom business metric (weighted profit) and want it logged to the SAME run that autologging created, alongside the automatically captured parameters and metrics. What should they do?

Accepted Answer

Call `mlflow.log_metric("weighted_profit", value)` inside the same `with mlflow.start_run()` block after the model is trained.

Answer

Start a second `mlflow.start_run()` block after training and call `mlflow.log_metric` there, since autologging closes the first run automatically.

Answer

Set `mlflow.sklearn.autolog(log_custom_metrics=True)` so the profit metric is captured automatically.

Answer

Disable autologging first, then manually log all parameters, metrics, and the custom metric to guarantee they land in one run.