Hard PMLE practice questions

Your team serves a product recommendation model on a Vertex AI online endpoint. Each request requires computing dense embeddings for a fixed catalog of 2 million products, then scoring the user against those embeddings. The catalog changes only once per day. Currently, product embeddings are recomputed on every request, causing p95 latency to exceed 800 ms and driving up GPU costs. You must reduce online latency while keeping recommendations fresh within one day. What is the best approach?

Your team trains an image classification CNN on a single NVIDIA A100 GPU using a per-device batch size of 64 and a base learning rate that converges well. To reduce wall-clock time, you scale to 8 A100 GPUs on one node using data-parallel synchronous training with tf.distribute.MirroredStrategy, keeping the same per-device batch size of 64. After the change, training loss decreases much more slowly per epoch and final accuracy drops compared to the single-GPU run. What is the most appropriate first adjustment?

Two data science teams share a Vertex AI Workbench environment to prototype churn models against a large BigQuery table. During prototyping, one team discovers that a colleague's earlier notebook produced a model with 4% higher AUC, but nobody can reproduce it because the underlying training data was overwritten by a nightly ETL job and the feature engineering code was iterated in place. Leadership asks you to establish a practice so future prototype results can be reliably reproduced and compared across teams. Which approach best addresses both data and model reproducibility?

A SaaS company stores millions of historical customer support tickets in BigQuery, each labeled with one of five routing categories (Billing, Technical, Account, Sales, Other). The data engineering team wants to automatically predict the routing category for new tickets using the text description column, and they prefer to build and serve the model without moving data out of BigQuery or writing custom training code. Which approach best meets these requirements?

You operate a deployed churn-prediction model on Vertex AI. Your monitoring shows that the statistical distribution of the incoming feature values has remained stable over the past three months (no input feature skew or drift detected). However, business stakeholders report that the model's live accuracy, measured against ground-truth churn labels that arrive 30 days later, has steadily declined from 0.88 to 0.71. What is the most likely cause of this degradation?

Your team is training a large image classification CNN on a single NVIDIA A100 GPU using TensorFlow. To fit a larger batch size and speed up training, you enabled mixed-precision (float16) training. After the change, throughput improved significantly, but the loss frequently becomes NaN early in training and the model fails to converge. What is the most appropriate action to fix the instability while keeping the performance benefits of mixed precision?

Your team is training a 40-billion-parameter transformer language model from scratch. Each model replica's parameters, optimizer states, and activations far exceed the 40 GB of memory available on a single GPU, so the full model cannot fit on one device. You have access to a multi-node cluster of GPUs on Vertex AI custom training. Which distributed training approach should you adopt to make training feasible?

A retail ML team preprocesses large volumes of transaction data for a churn model. During exploration in a Vertex AI Workbench notebook, engineers apply normalization and vocabulary-based categorical encoding using pandas. When the model is deployed, the online serving pipeline must apply the exact same transformations, but the team discovers subtle differences between training and serving preprocessing that degrade predictions. They want a scalable, reproducible approach that guarantees identical transformation logic in both batch training preprocessing and online serving. Which approach should they adopt?

Your team is training a model on a dataset of hospital patient records to predict readmission risk. Legal and compliance flag that the model must not allow attackers to infer whether a specific individual's record was part of the training set, even if the attacker has access to the model and some auxiliary data. Model accuracy is important but secondary to this privacy guarantee. Which approach best mitigates this specific privacy risk?

Your team serves a large BERT-based text classifier on a Vertex AI endpoint with GPU accelerators. Traffic has grown, and GPU serving costs are becoming unsustainable. The business requires that classification accuracy remain within 1% of the current model, and that p95 latency stay under 50 ms. You want to reduce serving cost the most while meeting both constraints, and you have access to the full labeled training dataset plus abundant unlabeled production text. Which optimization approach should you pursue first?

Your team has a TensorFlow model that trains on a 4 TB dataset stored in Cloud Storage as TFRecord files. A single-node 8-GPU machine takes over 40 hours per training run, which is blocking your experimentation velocity. The model uses large embedding layers and dense transformer blocks, and the training loop is written with the Keras API. You need to dramatically reduce wall-clock training time while keeping code changes minimal and controlling cost. What is the most appropriate approach?

You run a TensorFlow image-classification model on a Vertex AI online endpoint backed by GPUs. Traffic is bursty, and during peak periods GPU utilization stays low (around 30%) even though request latency climbs and clients report timeouts. Each individual request contains a single image. You want to increase throughput and improve GPU efficiency without reducing model accuracy or provisioning more replicas. What is the most effective change?

You deployed a gradient-boosted model on a Vertex AI endpoint to predict loan default. Overall prediction accuracy and the marginal distributions of each input feature (monitored via training-serving skew and drift detection) have stayed within alert thresholds for three months. However, business stakeholders report that recent predictions no longer match domain intuition, and a manual audit suggests the model is now weighting certain features very differently than at launch. Which monitoring approach would most directly detect this problem?

A media company deploys a recommendation model that ranks articles for users. The model is retrained weekly using logged user clicks as ground-truth labels. Over several months, the team notices that recommendation diversity collapses—users increasingly see only a narrow set of popular topics, and engagement among niche-interest users declines. Data drift and skew monitoring show no anomalies in input feature distributions. What is the most likely root cause of this degradation?

Your team deployed a resume-screening model to rank candidates for engineering roles. During a responsible AI review, you discover that although gender was explicitly removed from the training data, the model still recommends male candidates at a significantly higher rate. Investigation shows the model relies heavily on features like 'years of continuous employment' and membership in certain college sports clubs, which correlate strongly with gender. What is the MOST appropriate action to address this fairness risk?

Your team hosts 12 lightweight image-classification models on Vertex AI, each receiving low but steady traffic (a few requests per second). Each model fits comfortably in about 2 GB of GPU memory. Currently each model runs on its own dedicated node with a full NVIDIA T4 GPU, and GPU utilization per node hovers around 8%. Leadership wants to cut serving costs without materially increasing latency. What is the most effective way to reduce cost here?

Your team is running Vertex AI hyperparameter tuning on a deep learning model that takes about 6 hours per trial. You have a fixed budget of 40 total trials and want the best possible objective value. You are using Bayesian optimization as the search algorithm. Your manager wants results delivered as quickly as possible, but also wants the search to be as effective as possible. How should you configure the maxParallelTrials setting to balance wall-clock time and search quality?

A manufacturing quality-control model classifies product defects from images captured by factory-floor cameras. It performed well in production for six months. After the plant replaced all its cameras with a new higher-resolution model, engineers notice the defect classifier's precision has dropped sharply, though the physical products and defect types are unchanged. Ground-truth labels from manual inspection confirm the degradation is real. What is the MOST likely cause, and how should you address it?

You operate a deployed churn-prediction model for a subscription business. Input feature distributions monitored by Vertex AI Model Monitoring have remained stable for months, and there are no alerts on feature skew or feature attribution drift. However, the business team reports that model accuracy, measured against actual churn outcomes as labels arrive, has steadily declined over the last quarter. A recent pricing change altered which customers tend to churn. What is the most accurate characterization of what is happening, and what should you monitor going forward?

A retail company deployed a demand-forecasting model on Vertex AI six months ago. The team observes that prediction accuracy has gradually declined over the last two months, but Vertex AI Model Monitoring reports no significant feature drift or prediction drift alerts. The input feature distributions closely match the training baseline. Business stakeholders confirm that customer purchasing behavior has shifted due to a new competitor entering the market. What is the most likely cause of the performance degradation, and what should the team do?

You deployed a demand-forecasting model to a Vertex AI endpoint and enabled model monitoring for feature skew and drift. The business expects gradual, legitimate shifts in customer behavior over the coming months as a new product line ramps up. Your goal is to detect training-serving skew that indicates a data-pipeline problem, while avoiding a flood of alerts caused by the expected slow behavioral changes. How should you configure the monitoring baselines?

A data science team of six people at a healthcare analytics company needs to collaborate on model prototyping using Vertex AI Workbench notebooks. Their requirements are: each member should have an isolated compute environment, all notebook code must be version-controlled and peer-reviewed before merging, and no protected health information (PHI) should be persisted in notebook outputs committed to the repository. Which approach best satisfies all three requirements?

Your team serves a fraud-scoring model on a Vertex AI online endpoint. Median (p50) latency is well within your 40 ms SLO, but the p99 latency regularly spikes to 300 ms, causing occasional timeouts for the payment service. CPU utilization is moderate and autoscaling is not triggering. Investigation shows the spikes correlate with garbage collection pauses and large individual request payloads that occasionally arrive together. Which action most directly reduces the p99 tail latency without over-provisioning?

A retail demand-forecasting model deployed on a Vertex AI endpoint has been serving stable predictions for six months. Model Monitoring suddenly fires drift alerts for three input features on the same day: 'store_region' (categorical), 'promo_flag' (categorical), and 'weekly_units_sold_lag1' (numeric). Business metrics tied to the model have not degraded yet. Before triggering an expensive retrain, what is the most appropriate first troubleshooting step?

Your team trains a large PyTorch image model on Vertex AI using data-parallel synchronous SGD across multiple nodes, each with 8 NVIDIA A100 GPUs. Training is bottlenecked by the all-reduce gradient synchronization step, and network communication between worker nodes dominates step time. You need to reduce this communication overhead and improve overall training throughput without changing your model code significantly. What should you do?