Question

A data scientist has assembled all numeric and label-encoded categorical predictors into a single 'features' vector column using VectorAssembler. They now want to train a Spark ML DecisionTreeClassifier and ensure that the columns holding label-encoded categorical values (each with a small number of distinct integer codes) are treated as categorical splits rather than continuous thresholds. Which Spark ML stage should they add to the pipeline before the classifier?

Accepted Answer

VectorIndexer, configured with maxCategories, fit on the assembled features column to auto-identify and index categorical features

Answer

StringIndexer applied again to the assembled 'features' vector column

Answer

StandardScaler to normalize the assembled features so the tree can distinguish categorical columns

Answer

OneHotEncoder applied to the assembled 'features' vector column