Question

A data scientist has trained a single deep decision tree on a tabular dataset. The model achieves near-perfect accuracy on the training set but performs poorly and inconsistently on the validation set, and the results vary widely when the tree is retrained on slightly different data samples. The scientist wants to keep using tree-based models but reduce this instability without significantly increasing bias. Which ensemble approach is the MOST appropriate choice?

Accepted Answer

Use bagging (e.g., a random forest) to train many trees on bootstrap samples and average their predictions

Answer

Use boosting (e.g., AdaBoost) to sequentially train shallow trees that correct prior errors

Answer

Increase the maximum depth of the single tree to capture more patterns

Answer

Switch to a single linear regression model to reduce complexity