Question

A data scientist is building a churn prediction model on a Spark DataFrame that contains several numeric columns with missing values. They plan to impute missing values with the column mean, scale features with StandardScaler, and evaluate the model on a holdout set. To prevent data leakage while following best practices, what is the correct order of operations?

Accepted Answer

Split into train/test first, then compute imputation and scaling parameters on the training set only and apply them to both sets

Answer

Impute and scale the entire dataset first, then split into train/test sets

Answer

Compute imputation on the full dataset, split, then scale train and test independently

Answer

Split into train/test first, then compute separate imputation and scaling parameters on each set independently