ML Model Development

A data scientist is training a deep neural network on a sparse text-classification dataset using SageMaker. Training with plain stochastic gradient descent (SGD) at a fixed learning rate converges very slowly, and different features clearly need different effective learning rates because some tokens appear frequently while others are rare. The team wants an optimizer that adapts the per-parameter learning rate automatically to speed up convergence without extensive manual learning-rate tuning. Which optimizer change best addresses this need?

A data scientist trains a binary classifier to predict whether a customer will churn. Only 4% of customers in the dataset actually churn. The initial model reports 96% accuracy on the validation set, but the business complains that almost no real churners are being flagged. Which evaluation approach should the data scientist adopt to properly assess and compare model performance for this problem?

A machine learning engineer is training a deep neural network image classifier on SageMaker. Using a very large batch size to maximize GPU throughput, they observe that training loss converges quickly but validation accuracy is noticeably lower than a prior run that used a smaller batch size. Compute budget allows either approach. Which adjustment is MOST likely to improve generalization while keeping training stable?

A data scientist trains a binary classifier that flags manufacturing defects. The training set contains 4% defective units. The initial model achieves 96% accuracy but misses most defects during validation. Management requires that the model catch as many true defects as possible while keeping false alarms manageable, and they want a single metric that balances both concerns for model selection. Which evaluation metric should the team optimize?

A financial services team trains a gradient boosting classifier to predict loan default. Downstream systems consume the model's predicted probabilities directly as risk scores to set interest rates, so the numeric probabilities must reflect true likelihoods (e.g., a predicted 0.30 should default about 30% of the time). During evaluation, the model shows strong ranking performance with AUC of 0.88, but the business team reports the predicted probabilities appear systematically too high in the mid-range. Which evaluation step best addresses whether the probabilities can be trusted as risk scores?

A machine learning engineer is training a neural network in a SageMaker training job to classify support tickets into 12 mutually exclusive categories. The labels are provided as integer class indices (0 through 11) rather than one-hot encoded vectors. The final layer uses a softmax activation with 12 units. During training, the loss function raises a shape-mismatch error expecting the target to have the same shape as the model output. Which change resolves the error with the least preprocessing effort?

A security team wants to detect unusual patterns in network traffic logs to flag potential intrusions. The dataset contains millions of unlabeled records with numeric features such as packet count, byte volume, and connection duration. The team has no labeled examples of attacks and needs a scalable SageMaker built-in algorithm that can identify statistical outliers in this high-dimensional data. Which algorithm best fits this requirement?

A retail company wants to group its 2 million customers into distinct segments based on purchasing behavior, browsing patterns, and demographic attributes. The marketing team has no predefined labels for the segments and wants the model to discover natural groupings in the data so they can design targeted campaigns. Which SageMaker built-in algorithm is most appropriate for this task?

A retail company wants to forecast weekly product demand across 5,000 store-item combinations. The dataset includes historical sales, promotional flags, holiday calendars, and price changes. The team needs a single model that can produce probabilistic forecasts (with prediction intervals) for all series simultaneously and learn shared patterns across related items. Which SageMaker built-in algorithm best fits this requirement?

A retail company wants to build an image classifier that identifies which of 40 product categories appears in a photo. They have only about 3,000 labeled images total (roughly 75 per category) and limited GPU budget. The ML team needs the fastest path to a reasonably accurate model without collecting more labeled data. Which modeling approach best fits this situation?

A data science team at a real estate company needs to predict house sale prices from a tabular dataset containing 40 numeric and categorical features and about 120,000 rows. The relationships between features and price are highly non-linear, and the team wants strong predictive accuracy with minimal feature engineering while training efficiently on SageMaker. Which modeling approach should they choose?

A media company wants to test five different article headline variants on its homepage to maximize click-through rate. The traffic is high, and business stakeholders want to minimize the number of impressions wasted on clearly underperforming headlines while the test is running, dynamically shifting more traffic toward better-performing variants as evidence accumulates. Which modeling approach best fits this requirement?

A manufacturing company trains an XGBoost binary classifier in SageMaker to detect defective parts on an assembly line. Only 1.5% of parts are defective. The initial model achieves 98.5% accuracy but misses nearly all defective parts, which is unacceptable because undetected defects cause costly recalls. The team wants to improve the model's ability to catch defects while keeping the training pipeline simple and avoiding synthetic data generation. Which approach BEST addresses this problem?

A data science team must build a text-classification model to route incoming support tickets into 12 categories. They have only 400 labeled tickets total but millions of unlabeled historical tickets. Management wants the highest accuracy possible while minimizing new labeling effort. Which modeling approach best fits this situation?

A data science team trains a new version of a customer churn classifier weekly. Each version is registered as a new model package version in an existing SageMaker Model Registry model package group. Before promoting a new version to production, the team must compare its offline evaluation metrics (AUC, F1) against the currently approved production version and record the decision, keeping a full history of which version was approved and by whom. Which approach best meets these requirements with the least custom development?

A data science team trains two versions of a binary classifier on the same small validation set of 400 records. Version A reports 88.0% accuracy and Version B reports 88.5% accuracy across a single evaluation run. The lead engineer must decide whether Version B is genuinely better before promoting it in the SageMaker Model Registry. What is the most statistically sound next step?

A data scientist trains a binary classifier to detect fraudulent transactions. Fraud represents only 1.5% of all transactions. On the validation set, the model achieves 98.5% accuracy, but the business reports that many fraudulent transactions still slip through undetected. The confusion matrix shows a very high number of false negatives and very few false positives. Which action will BEST address the business concern?

A data scientist is training a linear model with gradient descent on a dataset where one feature ranges from 0 to 1 and another ranges from 0 to 1,000,000. During training, the loss oscillates and takes many epochs to converge, and reducing the learning rate slows progress to a crawl without fixing the instability. What is the MOST likely cause and the appropriate remedy?

A data scientist is building a document deduplication system. Each document is represented as a high-dimensional TF-IDF embedding vector, but document lengths vary widely, so longer documents produce vectors with larger magnitudes. The team wants a similarity metric for a k-nearest-neighbors model that focuses on the orientation (topical content) of vectors rather than their magnitude, so that a short and a long document about the same topic are still judged similar. Which distance/similarity measure best fits this requirement?

A machine learning engineer is building a neural network image classifier in SageMaker that must output calibrated class probabilities so a downstream business rule can act only when confidence exceeds 90%. During training, the engineer needs to choose a loss function for the final softmax output layer. Which loss function best supports producing well-calibrated probability estimates for this multiclass problem?