Question

A data scientist is using Spark ML's CrossValidator to tune a RandomForestClassifier. Each time they rerun the tuning notebook on the same dataset and identical parameter grid, they observe slightly different average metrics for the same hyperparameter combinations, making it hard to compare runs. What is the most likely cause and the correct fix?

Accepted Answer

The CrossValidator's fold assignment is randomized each run; set the seed parameter on CrossValidator to make the fold splits deterministic.

Answer

The evaluator metric changes automatically between runs; explicitly set metricName on the evaluator so results are consistent.

Answer

The parameter grid is being reordered each run; sort the estimatorParamMaps list before passing it to CrossValidator.

Answer

CrossValidator caches stale metrics; call collectSubModels=True to force recomputation on each run.