Question

A data scientist is tuning an XGBoost model with 5 continuous hyperparameters, each spanning a wide range. They have a limited compute budget and want to find strong hyperparameter values with far fewer trials than an exhaustive approach would require. Which tuning strategy is the BEST fit, and why?

Accepted Answer

Use Hyperopt with the Tree-structured Parzen Estimator (TPE) algorithm, because it uses results from prior trials to choose promising hyperparameter values, converging with fewer evaluations.

Answer

Use an exhaustive grid search, because evaluating every combination guarantees the global optimum within the budget.

Answer

Use pure random search with a fixed large number of trials, because random sampling always outperforms adaptive methods for continuous spaces.

Answer

Use Hyperopt with the random sampling (rand.suggest) algorithm, because it adapts the search distribution based on prior trial losses.