Question

A data scientist is building a Spark ML pipeline to predict customer churn. The pipeline contains a StringIndexer, a VectorAssembler, and a LogisticRegression stage. After calling pipeline.fit(trainingDF), the data scientist wants to generate predictions on a separate testDF. Which statement correctly describes the objects involved?

Accepted Answer

pipeline.fit() returns a PipelineModel (a Transformer); calling its transform(testDF) method produces predictions.

Answer

pipeline.fit() returns a Pipeline (an Estimator); calling its fit(testDF) method produces predictions.

Answer

pipeline.fit() returns a PipelineModel (an Estimator); calling its transform(testDF) refits the model on the test data.

Answer

pipeline.fit() returns a DataFrame directly containing the predictions, so no further calls are needed on testDF.