Question

A data science team at a biotech firm has a labeled dataset of only 1,800 records for a tabular classification task predicting patient treatment response. They are prototyping several model architectures and want a reliable estimate of generalization performance before committing to a full training and hyperparameter tuning run on Vertex AI. A single hold-out validation split gave them wildly different accuracy numbers each time they reshuffled the data. What is the most appropriate approach to obtain a robust performance estimate given the dataset size?

Accepted Answer

Use k-fold cross-validation (e.g., 5- or 10-fold) to average performance across multiple train/validation partitions.

Answer

Increase the size of the single hold-out validation set to 50% of the data to reduce variance in the estimate.

Answer

Train on all 1,800 records and report training accuracy as the generalization estimate.

Answer

Collect a larger dataset before doing any modeling, since 1,800 records cannot yield meaningful estimates.