Question

A data scientist is training a gradient boosting model on a dataset with over 5 million rows. Model training takes roughly 45 minutes per fit on the assigned cluster. The scientist needs a reliable estimate of generalization performance to compare a small number of candidate configurations, but wants to keep total compute time manageable. Which validation strategy is the most appropriate for this situation?

Accepted Answer

Use a single train/validation/test split (holdout), evaluating each candidate on the fixed validation set

Answer

Use 10-fold cross-validation to maximize the reliability of the performance estimate

Answer

Use leave-one-out cross-validation so every row contributes to validation

Answer

Skip a validation set and report performance on the training data directly