Model Development

A data scientist is building a regression model on a small dataset of only 300 labeled records. A single 80/20 train/test split produces validation metrics that vary widely each time the seed changes, making it hard to trust the model's estimated performance. The scientist wants a more reliable estimate of generalization error before registering the model. Which change to the training/validation strategy best addresses this problem?

A data scientist has a Spark DataFrame containing sales data for 500 retail stores, with columns store_id, week, features, and target. They want to train a separate scikit-learn regression model for each store in parallel across the cluster, returning a DataFrame with one row per store containing the store_id and its model's RMSE. Which approach correctly accomplishes this in a single distributed operation?

A data scientist is tuning hyperparameters for a scikit-learn gradient boosting model on a single large Databricks cluster. The training dataset fits comfortably in the driver's memory, and each individual model trains quickly on one machine. They want to use Hyperopt to evaluate many hyperparameter combinations and distribute those evaluations across the cluster's worker nodes to finish the search faster. Which Hyperopt configuration should they use in the fmin() call?

A data scientist is using Hyperopt's fmin() with the TPE algorithm to tune a scikit-learn classifier. The objective is to maximize validation accuracy. Inside the objective function, the model is trained and cross-validated, producing a mean accuracy score stored in a variable named acc. What should the objective function return so that Hyperopt tunes the model correctly?

A data scientist is tuning an XGBoost model on a single-node Databricks cluster using Hyperopt. They define a search space, an objective function that returns a dictionary with keys 'loss' and 'status', and call fmin() with the TPE algorithm. They want the optimization to try exactly 60 different hyperparameter configurations before stopping. Which fmin() argument controls this behavior?

A data scientist is using Hyperopt to tune a gradient boosting model. They want to search for the learning rate over a range from 0.001 to 0.1, and they know that this parameter's effect is multiplicative rather than additive—values like 0.001, 0.01, and 0.1 should each get comparable exploration. Which search space definition should they use for the learning rate?

A data scientist is tuning an XGBoost model with 5 continuous hyperparameters, each spanning a wide range. They have a limited compute budget and want to find strong hyperparameter values with far fewer trials than an exhaustive approach would require. Which tuning strategy is the BEST fit, and why?

A data scientist runs Hyperopt's fmin() on a single-node cluster to tune an XGBoost model, passing a Trials() object as the trials argument. After optimization completes, they want to programmatically inspect the loss and hyperparameters tried for every evaluation, not just the best result returned by fmin(). Which approach lets them access the full history of all trials?

A data scientist has been prototyping feature engineering using a pandas DataFrame in a Databricks notebook. The dataset has grown to 400 GB and no longer fits in the driver's memory, causing out-of-memory errors. The scientist wants to keep the existing pandas-style code (e.g., df.groupby(...).mean(), df['col'].fillna(...)) with minimal rewriting while distributing the computation across the cluster. Which approach best meets this requirement?

A data scientist has a 2 GB training dataset that fits comfortably in the memory of a single Databricks worker. They want to train a gradient-boosted tree model using scikit-learn and are unsure whether to convert their Spark DataFrame to a pandas DataFrame or keep the data distributed. Which statement correctly describes the most appropriate approach for this situation?

A data scientist has a Spark DataFrame containing sales data for 500 different stores, identified by a store_id column. They want to train a separate lightweight scikit-learn regression model for each store in parallel across the cluster, returning a single DataFrame that summarizes each store's model coefficients. Which approach is the most appropriate?

A data scientist is building a regression model to predict housing prices in dollars. Stakeholders have stated that large prediction errors are especially costly and should be penalized more heavily than small ones, and they want the error reported in the same units as the target (dollars). Which evaluation metric best satisfies these requirements?

A data scientist trains a binary classifier to predict customer churn. The business wants to evaluate how well the model ranks customers by churn probability across all possible decision thresholds, because the exact classification threshold has not yet been decided and may change over time. Which evaluation metric best supports this requirement?

A data scientist trains a scikit-learn binary classifier to detect fraudulent transactions. Only about 2% of transactions are fraudulent, and the cost of missing a fraudulent transaction (false negative) is far higher than the cost of flagging a legitimate one (false positive). Which single evaluation metric should the data scientist prioritize when comparing candidate models?

A data scientist is evaluating a scikit-learn RandomForestClassifier on a dataset that comfortably fits in memory on a single node. They run cross_val_score(model, X, y, cv=5, scoring='f1') and want to report a single, robust estimate of the model's generalization performance for stakeholders. Which value should they report?

A data scientist is training a scikit-learn model on a single-node cluster. Their workflow first applies a StandardScaler to the full training set, then runs GridSearchCV with 5-fold cross-validation on the scaled data. A reviewer flags that this approach can produce overly optimistic validation scores. What is the correct way to restructure the code so that scaling is properly incorporated into the cross-validation process?

A data scientist is tuning a scikit-learn GradientBoostingClassifier on a single-node cluster. The search space includes 5 continuous hyperparameters, each with many candidate values, and the team has a strict wall-clock time budget. They want to explore the space efficiently and still have a reasonable chance of finding a strong configuration. Which tuning approach best fits these constraints?

A data scientist has a training dataset of 2 GB that fits comfortably in the memory of a single Databricks driver node. They want to train a scikit-learn RandomForestClassifier while still distributing the hyperparameter search across the workers of the cluster. Which approach best accomplishes this goal?

A data scientist has a training dataset of 800 MB that comfortably fits into the memory of a single Databricks worker node. They want to train a scikit-learn RandomForestClassifier and perform a large hyperparameter search over 500 candidate configurations. They want to distribute the individual training trials across the cluster's worker nodes to speed up the search. Which approach best meets this requirement?

A data scientist trains a Spark ML LogisticRegression model to predict customer churn (binary label). The dataset is heavily imbalanced, with only 8% of customers churning. They want to evaluate model performance in a way that is robust to this class imbalance and does not depend on a fixed classification threshold. Which evaluator and metric configuration should they use?