Professional Machine Learning Engineer

Question 1

A retail chain wants to classify products on store shelves using cameras that operate in stores with unreliable internet connectivity. The data science team has limited ML coding experience but has thousands of labeled product images. They need a solution that trains a custom image classifier and can run inference directly on in-store edge devices without a constant network connection. Which approach best meets these requirements?

Accepted Answer

Train a custom model with Vertex AI AutoML Image Classification using the 'Edge' model type, then export the model to run on the in-store devices.

Answer

Use the Cloud Vision API label detection endpoint from each in-store device to classify products in real time.

Answer

Train a Vertex AI AutoML Image Classification model with the 'Cloud' hosted model type and call its online prediction endpoint from the devices.

Answer

Build a custom TensorFlow CNN in Vertex AI Workbench notebooks and containerize it for edge deployment.

Question 2

A fintech company wants to build a binary classification model in Vertex AI AutoML Tabular to detect fraudulent transactions. Only 0.8% of the labeled transactions are fraudulent. The team's priority is to catch as many actual fraud cases as possible while keeping false positives at a manageable level, and they have limited ML expertise. Which approach should they configure to best meet this goal?

Accepted Answer

Train the AutoML Tabular model and set the optimization objective to maximize the area under the precision-recall curve (AU PRC).

Answer

Train the AutoML Tabular model with the default optimization objective of maximizing the area under the ROC curve (AU ROC).

Answer

Manually downsample the majority class to a 50/50 balance before training and optimize for overall accuracy.

Answer

Switch to a BigQuery ML logistic regression model and optimize for log loss since AutoML cannot handle imbalanced data.

Question 3

A logistics company wants to predict package delivery times using a structured dataset in Vertex AI. The data science team has limited coding experience and needs a solution that requires minimal ML expertise. Business stakeholders also require the model to report which features most influence each individual prediction, and the model must be queried in real time from a customer-facing web application. Which approach best meets all requirements?

Accepted Answer

Train an AutoML tabular regression model in Vertex AI, deploy it to an online endpoint, and enable Vertex Explainable AI feature attributions on the predictions.

Answer

Train a custom TensorFlow regression model with a hand-written training loop and serve it via a Compute Engine VM.

Answer

Train an AutoML tabular regression model in Vertex AI and use only batch prediction jobs to generate delivery time estimates.

Answer

Use BigQuery ML to train a linear regression model and export it to a TensorFlow SavedModel for local inference.

Question 4

A marketing analytics team with limited ML expertise wants to build a churn prediction model from a customer table already in BigQuery. They choose Vertex AI AutoML Tabular through the console. During job configuration, they must set a training budget expressed in node hours. They ask you how this setting affects the outcome and how to choose a reasonable value for their first experiment. What is the most accurate guidance?

Accepted Answer

The budget caps the maximum compute AutoML spends searching architectures and tuning; AutoML may stop early if the model converges, and a small budget is appropriate for an initial baseline experiment.

Answer

The budget sets the number of prediction nodes for the deployed endpoint, so it should match expected serving traffic rather than training needs.

Answer

The budget determines how many rows of the dataset AutoML uses for training, so a higher budget always means more data is included.

Answer

The budget is a hard minimum runtime; AutoML will always consume the entire allocation even if the model has already converged, so it should be set as low as possible to save money.

Question 5

A sports media company wants to automatically tag short game clips with the specific athletic actions they contain (e.g., 'shot on goal', 'free kick', 'tackle') along with the time segments where each action occurs. The data science team has only a few weeks, limited ML expertise, and a labeled dataset of a few thousand video clips. They want the lowest-code approach on Google Cloud that produces temporal action segments. Which solution best fits these requirements?

Accepted Answer

Use AutoML (Vertex AI) to train an action recognition video model on the labeled clips

Answer

Use the Vision API's label detection endpoint on sampled frames from each clip

Answer

Write a custom PyTorch 3D-CNN training job on Vertex AI custom training with a GPU cluster

Answer

Use the Video Intelligence API's prebuilt label detection to identify actions

Question 6

Your team serves a product recommendation model on a Vertex AI online endpoint. Each request requires computing dense embeddings for a fixed catalog of 2 million products, then scoring the user against those embeddings. The catalog changes only once per day. Currently, product embeddings are recomputed on every request, causing p95 latency to exceed 800 ms and driving up GPU costs. You must reduce online latency while keeping recommendations fresh within one day. What is the best approach?

Accepted Answer

Precompute all product embeddings in a nightly batch prediction job, store them in a low-latency store (e.g., Vertex Feature Store or a vector index), and have the online endpoint only compute the user embedding and perform the lookup/scoring at request time.

Answer

Increase the online endpoint's minimum replica count and attach larger GPUs so the per-request embedding computation completes faster.

Answer

Enable dynamic request batching on the online endpoint so multiple user requests share the product-embedding computation.

Answer

Apply post-training int8 quantization to the embedding model so each per-request embedding computation runs faster on the same hardware.

Question 7

A retail analytics team needs to score 400 million customer records once per week using a trained TensorFlow model deployed on Vertex AI. The records live in a large BigQuery table. Latency is not a concern, but the team wants the job to complete as quickly and cheaply as possible while avoiding the operational overhead of managing infrastructure. Which approach best meets these requirements?

Accepted Answer

Run a Vertex AI batch prediction job that reads directly from the BigQuery table as the input source and writes results back to BigQuery, letting the service distribute the workload.

Answer

Deploy the model to a Vertex AI online endpoint with autoscaling and send the 400 million records as individual prediction requests from a Dataflow job.

Answer

Export the BigQuery table to a single large CSV file in Cloud Storage and run a batch prediction job pointed at that file.

Answer

Provision a large GPU-backed Compute Engine VM, load the model, and iterate through the records with a custom Python loop.

Question 8

A retail analytics team needs to score their entire customer base (about 40 million records) once every night to generate propensity scores for a next-day marketing campaign. The scores are consumed by a downstream BigQuery table the following morning, so end-to-end latency of a few hours is acceptable. The team currently keeps a Vertex AI online endpoint with multiple GPU replicas running 24/7 to handle this workload, but finance has flagged the cost as excessive. What is the most appropriate change to reduce cost while meeting requirements?

Accepted Answer

Switch to a Vertex AI batch prediction job that reads from BigQuery, runs on demand each night, and provisions compute only for the duration of the job.

Answer

Keep the online endpoint but reduce it to a single CPU replica and disable autoscaling to lower hourly cost.

Answer

Keep the online endpoint but enable dynamic request batching to increase throughput per replica.

Answer

Deploy the model to a private endpoint inside the VPC to reduce network egress charges during scoring.

Question 9

Your team maintains a fraud-scoring model deployed on a Vertex AI online endpoint. A new business requirement asks you to re-score the entire historical transaction table (2 billion rows) once, to backfill risk labels for an analytics dashboard. There is no latency requirement for this one-time job, but you want to minimize cost and total wall-clock time. The online endpoint currently autoscales between 3 and 10 GPU replicas. What is the most appropriate approach?

Accepted Answer

Run a Vertex AI batch prediction job that reads the table from BigQuery, using a high machine count to parallelize the workload.

Answer

Send the 2 billion rows to the existing online endpoint by fanning out many concurrent HTTP requests and temporarily raising max replicas to 50.

Answer

Deploy a second online endpoint dedicated to the backfill and stream all rows through it with dynamic batching enabled.

Answer

Export the online endpoint model and run inference locally in a single notebook loop over all 2 billion rows.

Question 10

Your team trains an image classification CNN on a single NVIDIA A100 GPU using a per-device batch size of 64 and a base learning rate that converges well. To reduce wall-clock time, you scale to 8 A100 GPUs on one node using data-parallel synchronous training with tf.distribute.MirroredStrategy, keeping the same per-device batch size of 64. After the change, training loss decreases much more slowly per epoch and final accuracy drops compared to the single-GPU run. What is the most appropriate first adjustment?

Accepted Answer

Increase the learning rate to account for the larger effective (global) batch size, for example by applying linear scaling relative to the batch-size increase.

Answer

Reduce the per-device batch size to 8 so the global batch size matches the original single-GPU batch size of 64.

Answer

Switch from MirroredStrategy to MultiWorkerMirroredStrategy to improve gradient synchronization across the GPUs.

Answer

Disable synchronous training and use asynchronous parameter-server updates to speed up convergence.

Question 11

A data science team prototypes models in a Vertex AI Workbench notebook using a 50 GB sample pulled into a Pandas DataFrame. The full training dataset lives in BigQuery and is roughly 8 TB, requiring complex windowed aggregations and joins across several tables. The team wants a preprocessing approach that scales to the full dataset for production training runs, minimizes custom infrastructure management, and produces feature tables that both training and future batch jobs can reuse. Which approach should they adopt?

Accepted Answer

Perform the set-based aggregations and joins directly in BigQuery using SQL, materializing the results into curated feature tables that downstream jobs query.

Answer

Increase the notebook machine to a high-memory instance and continue running the transformations with Pandas on the full 8 TB dataset.

Answer

Export the BigQuery tables to CSV files in Cloud Storage and process them with a manually configured Dataproc cluster running Spark.

Answer

Load the full dataset into the notebook in chunks and loop over each chunk with scikit-learn preprocessing transformers.

Question 12

Two data science teams share a Vertex AI Workbench environment to prototype churn models against a large BigQuery table. During prototyping, one team discovers that a colleague's earlier notebook produced a model with 4% higher AUC, but nobody can reproduce it because the underlying training data was overwritten by a nightly ETL job and the feature engineering code was iterated in place. Leadership asks you to establish a practice so future prototype results can be reliably reproduced and compared across teams. Which approach best addresses both data and model reproducibility?

Accepted Answer

Snapshot each training dataset as a BigQuery table with a timestamp suffix, log each run to Vertex AI Experiments capturing parameters and metrics, and register resulting models in Vertex AI Model Registry with links to the dataset snapshot.

Answer

Have each team export their notebook to a shared Cloud Storage bucket after every run and rely on the notebook cell outputs to document metrics and data used.

Answer

Enable BigQuery table expiration on the ETL output tables so historical data is retained longer, and email AUC results to the group after each experiment.

Answer

Increase the Workbench instance disk size so all intermediate CSV files are stored locally, and use Git commit messages to record which model achieved the best AUC.

Question 13

A data science team prototypes models in Vertex AI Workbench notebooks. Their preprocessing code uses pandas to read and transform a 400 GB BigQuery table, but the notebook kernel repeatedly crashes with out-of-memory errors. The team wants to keep writing familiar pandas-style code in the notebook while pushing the heavy computation down to BigQuery so the data never has to fit in the notebook's memory. What should they do?

Accepted Answer

Use the BigQuery DataFrames (bigframes) library, which exposes a pandas-compatible API but executes operations as BigQuery SQL on the server side.

Answer

Increase the Workbench instance to the largest available machine type with more RAM and continue using pandas with the BigQuery client.

Answer

Export the BigQuery table to CSV files in Cloud Storage and read them into pandas in chunks with the chunksize parameter.

Answer

Load the table into a Vertex AI Feature Store and read features into a pandas DataFrame for preprocessing.

Question 14

A data science team at a logistics company needs to explore three years of shipment event logs stored as newline-delimited JSON files in a Cloud Storage bucket (roughly 40 TB, organized in Hive-style date directories like /year=2023/month=05/). Analysts want to run ad hoc exploratory SQL in BigQuery to profile the data before committing to any modeling approach, but leadership is worried about query costs and does not want to duplicate the raw data into BigQuery-managed storage yet. Which approach best supports low-cost, iterative exploration?

Accepted Answer

Create a BigQuery external table over the Cloud Storage files with Hive partitioning enabled, and have analysts filter on the partition columns in their exploratory queries.

Answer

Load all 40 TB into a native BigQuery table with ingestion-time partitioning so analysts can query without touching Cloud Storage.

Answer

Create a non-partitioned external table over the JSON files and instruct analysts to add LIMIT clauses to reduce the amount of data scanned.

Answer

Use a Dataflow pipeline to convert the JSON to Parquet in Cloud Storage first, then query each file individually through the BigQuery Storage Read API.

Question 15

A retail analytics team stores all their customer transaction data in BigQuery. They want to build a churn prediction model directly in BigQuery ML using a boosted tree classifier, and they want BigQuery ML to automatically search for the best combination of learning rate and max tree depth rather than manually experimenting. They also need the final model to be usable for online predictions from a Vertex AI endpoint later. Which approach best meets these requirements with the least custom code?

Accepted Answer

Create a BOOSTED_TREE_CLASSIFIER model in BigQuery ML with NUM_TRIALS and HPARAM_RANGE/HPARAM_CANDIDATES options to enable hyperparameter tuning, then export the model and register it to Vertex AI for online serving.

Answer

Train a custom XGBoost model in a Vertex AI custom training job, manually tune hyperparameters with Vertex AI Vizier, then deploy to an endpoint.

Answer

Use AutoML Tables in Vertex AI, which handles hyperparameter tuning automatically, and skip BigQuery ML entirely.

Answer

Create a LOGISTIC_REG model in BigQuery ML with L1_REG and L2_REG regularization options, since boosted trees do not support hyperparameter tuning in BigQuery ML.

Question 16

A retail analytics team stores three years of daily sales transactions in BigQuery. They need to produce a 30-day demand forecast for each of 5,000 products, including confidence intervals, while keeping the data in place and minimizing custom code or infrastructure management. Which approach best meets these requirements?

Accepted Answer

Create a BigQuery ML ARIMA_PLUS model with a time-series column, item identifier, and TIME_SERIES_ID_COL to train per-product models, then use ML.FORECAST to generate predictions with prediction intervals.

Answer

Export the sales data to Cloud Storage, train a custom LSTM model on Vertex AI with a TensorFlow container, and deploy it to an endpoint for batch forecasting.

Answer

Use a BigQuery ML LINEAR_REG model with the date as a numeric feature and call ML.PREDICT to estimate future sales for each product.

Answer

Send each product's historical sales to the Vertex AI Gemini API and prompt the model to predict the next 30 days of demand.

Question 17

Your team trains a large PyTorch image model on Vertex AI custom training. Each run takes about 30 hours on a single node with multiple GPUs. To reduce cost, you switched to Spot (preemptible) VMs, but you notice that when a VM is reclaimed, the job restarts from scratch and wastes many GPU-hours. You must keep costs low while ensuring interrupted runs can resume efficiently. What should you do?

Accepted Answer

Periodically save model and optimizer state to Cloud Storage and configure the training job to resume from the latest checkpoint on restart, keeping Spot VMs and enabling automatic retries.

Answer

Switch back to standard (on-demand) VMs and remove checkpointing since the job will no longer be interrupted.

Answer

Reduce the number of training epochs so that each run completes before a Spot VM is likely to be preempted.

Answer

Move the training to a smaller CPU-only machine so that preemption happens less frequently and restarts are cheaper.

Question 18

You operate a deployed churn-prediction model on Vertex AI. Your monitoring shows that the statistical distribution of the incoming feature values has remained stable over the past three months (no input feature skew or drift detected). However, business stakeholders report that the model's live accuracy, measured against ground-truth churn labels that arrive 30 days later, has steadily declined from 0.88 to 0.71. What is the most likely cause of this degradation?

Accepted Answer

Concept drift — the relationship between the input features and the target label has changed even though input distributions are unchanged

Answer

Training-serving skew caused by inconsistent preprocessing between the training pipeline and the serving endpoint

Answer

Prediction drift caused by a shift in the distribution of the model's output scores

Answer

Data drift caused by a change in the input feature distributions over time

Question 19

Your team is training a large image classification CNN on a single NVIDIA A100 GPU using TensorFlow. To fit a larger batch size and speed up training, you enabled mixed-precision (float16) training. After the change, throughput improved significantly, but the loss frequently becomes NaN early in training and the model fails to converge. What is the most appropriate action to fix the instability while keeping the performance benefits of mixed precision?

Accepted Answer

Apply dynamic loss scaling so small gradients are not lost to float16 underflow while keeping numerically sensitive operations in float32.

Answer

Revert entirely to float32 training since mixed precision is inherently unstable on A100 GPUs.

Answer

Reduce the learning rate to near zero to prevent the loss from ever spiking to NaN.

Answer

Switch the hardware to a TPU v3 pod, which does not experience float16 numerical issues.

Question 20

Your team has prototyped a gradient-boosted decision tree model (XGBoost) on a small tabular dataset for predicting equipment maintenance needs. You now need to scale training to a 200 GB dataset and later serve batch predictions nightly. A junior engineer suggests provisioning a cluster of GPU-accelerated VMs for both training and inference to maximize speed. What is the most cost-effective and appropriate hardware recommendation for this workload?

Accepted Answer

Use high-memory CPU machines for both distributed training and batch inference, since tree-based models like XGBoost are largely CPU-bound and gain little from GPU acceleration relative to cost.

Answer

Use TPUs for training and GPUs for batch inference, because TPUs offer the best price-performance for all tabular model types.

Answer

Use a large fleet of GPU VMs for training and inference to match the engineer's recommendation, as GPUs accelerate all matrix operations regardless of model architecture.

Answer

Use TPU Pods for training because the 200 GB dataset requires the parallelism only TPUs provide, then serve inference on CPUs.

Question 21

Your team deployed a credit-approval model on Vertex AI. Regulators require ongoing evidence that the model does not produce discriminatory outcomes across protected groups such as gender and ethnicity. The model shows strong overall accuracy, but leadership is concerned about fairness in production. Which approach best addresses this responsible-AI risk during ongoing monitoring?

Accepted Answer

Continuously monitor approval rates and error rates disaggregated by protected group using fairness metrics such as demographic parity difference and equal opportunity difference, and alert when disparities exceed defined thresholds.

Answer

Rely on the model's overall accuracy and AUC in production, since a high-performing model is inherently fair across all subgroups.

Answer

Remove the protected attributes from the training features so the model cannot use them, which guarantees the outcomes are unbiased.

Answer

Enable feature-attribution drift monitoring on the input features and treat any drift alert as evidence of a fairness violation.

Question 22

A data science team at a biotech firm has a labeled dataset of only 1,800 records for a tabular classification task predicting patient treatment response. They are prototyping several model architectures and want a reliable estimate of generalization performance before committing to a full training and hyperparameter tuning run on Vertex AI. A single hold-out validation split gave them wildly different accuracy numbers each time they reshuffled the data. What is the most appropriate approach to obtain a robust performance estimate given the dataset size?

Accepted Answer

Use k-fold cross-validation (e.g., 5- or 10-fold) to average performance across multiple train/validation partitions.

Answer

Increase the size of the single hold-out validation set to 50% of the data to reduce variance in the estimate.

Answer

Train on all 1,800 records and report training accuracy as the generalization estimate.

Answer

Collect a larger dataset before doing any modeling, since 1,800 records cannot yield meaningful estimates.

Question 23

Your team is training a 40-billion-parameter transformer language model from scratch. Each model replica's parameters, optimizer states, and activations far exceed the 40 GB of memory available on a single GPU, so the full model cannot fit on one device. You have access to a multi-node cluster of GPUs on Vertex AI custom training. Which distributed training approach should you adopt to make training feasible?

Accepted Answer

Model/tensor parallelism (e.g., sharding layers and parameters across GPUs), optionally combined with data parallelism, so a single model instance spans multiple devices

Answer

Standard synchronous data parallelism, replicating the full model on each GPU and averaging gradients across replicas

Answer

Reduce the global batch size to 1 and use gradient accumulation so the model fits on a single GPU

Answer

Asynchronous parameter-server training with each worker holding a full model copy and pushing updates independently

Question 24

Your ML team needs to preprocess 8 TB of raw clickstream logs stored in Cloud Storage before training. The pipeline must join the logs against a 200 GB user-profile table in BigQuery, apply session-based windowing, and write cleaned features back to BigQuery for downstream training. Multiple teams will reuse this pipeline on a scheduled daily basis. Which approach best satisfies scalability, reusability, and maintainability requirements?

Accepted Answer

Build an Apache Beam pipeline on Dataflow that reads logs from Cloud Storage, joins with the BigQuery user-profile data as a side input or via a BigQuery source, applies session windows, and writes to BigQuery, deployed as a reusable Flex Template.

Answer

Load all data into a single BigQuery table and perform the joins and windowing with a scheduled SQL query, exporting results to a CSV in Cloud Storage.

Answer

Write a Python script in a Vertex AI notebook using pandas to load the logs and profiles into memory, perform the joins, and upload the results to BigQuery.

Answer

Use a Dataproc Spark job triggered manually each day, storing intermediate results in HDFS and copying the final output to BigQuery.

Question 25

Your team runs a nightly Dataflow batch pipeline that reads raw event logs from Cloud Storage, applies transformations, and writes the results to a BigQuery table used by three other teams for feature exploration. Recently, downstream data scientists complain that record counts vary between runs even when the input files are identical, and they see duplicate rows after failed-and-retried jobs. You need to guarantee that repeated pipeline runs produce a consistent, deduplicated table without manual cleanup. What should you do?

Accepted Answer

Configure the BigQueryIO write with WRITE_TRUNCATE disposition and a deterministic key so each run replaces the table contents idempotently.

Answer

Switch the pipeline to streaming mode and rely on Pub/Sub exactly-once delivery to prevent duplicates.

Answer

Increase the number of Dataflow workers so the job finishes faster and is less likely to be retried.

Answer

Add a post-processing scheduled query in BigQuery that runs SELECT DISTINCT into a new table each morning.

Question 26

Your data engineering and ML teams share a clickstream pipeline. Raw events arrive continuously on a Pub/Sub topic and must be cleaned, windowed into 5-minute session aggregates, and written to BigQuery so both teams can build features from the same processed data. The processing logic includes stateful sessionization and must handle late-arriving events with watermarks. Which approach best meets these requirements?

Accepted Answer

Build an Apache Beam streaming pipeline on Dataflow that reads from Pub/Sub, applies session windows with allowed lateness, and streams results to BigQuery.

Answer

Schedule an hourly BigQuery scheduled query that pulls raw events from an external Pub/Sub table and aggregates them into sessions.

Answer

Use a Cloud Function triggered per Pub/Sub message to compute session aggregates and insert rows into BigQuery.

Answer

Export Pub/Sub messages to Cloud Storage in batches and run a nightly Dataproc Spark batch job to compute session windows.

Question 27

A retail ML team preprocesses large volumes of transaction data for a churn model. During exploration in a Vertex AI Workbench notebook, engineers apply normalization and vocabulary-based categorical encoding using pandas. When the model is deployed, the online serving pipeline must apply the exact same transformations, but the team discovers subtle differences between training and serving preprocessing that degrade predictions. They want a scalable, reproducible approach that guarantees identical transformation logic in both batch training preprocessing and online serving. Which approach should they adopt?

Accepted Answer

Use TensorFlow Transform (tf.Transform) with Dataflow to compute preprocessing statistics over the full dataset and export a transform graph that is embedded into the serving signature.

Answer

Rewrite the pandas preprocessing as a Cloud Function that both the training job and the serving endpoint call over HTTP before inference.

Answer

Save the pandas preprocessing code in a shared Git repository and require both the training and serving teams to import the same module.

Answer

Precompute all transformed features in BigQuery and store both training and serving features in Vertex AI Feature Store without embedding transformations in the model.

Question 28

You are an ML engineer joining a large enterprise with hundreds of BigQuery datasets spread across multiple projects owned by different business units. Before prototyping a churn model, you need to discover which existing tables contain customer transaction data, understand their schemas, check data quality metrics, and confirm you have appropriate access—without manually contacting each team. Which Google Cloud approach best enables this organization-wide data exploration and governance?

Accepted Answer

Use Dataplex to organize the datasets into lakes and zones, and use its integrated Data Catalog to search metadata, view schemas, and inspect data profiling and quality scans across projects.

Answer

Export every BigQuery table's schema to a shared Google Sheet and manually annotate which tables contain customer data for the team to reference.

Answer

Grant yourself BigQuery Admin on all projects, then run SELECT * queries against each table to inspect the data and infer its contents.

Answer

Provision a Vertex AI Feature Store and ingest all candidate tables so you can browse feature metadata in one place before modeling.

Question 29

Your team is training a model on a dataset of hospital patient records to predict readmission risk. Legal and compliance flag that the model must not allow attackers to infer whether a specific individual's record was part of the training set, even if the attacker has access to the model and some auxiliary data. Model accuracy is important but secondary to this privacy guarantee. Which approach best mitigates this specific privacy risk?

Accepted Answer

Apply differential privacy during training (e.g., DP-SGD) to bound the influence any single training example has on the model.

Answer

Encrypt the training dataset at rest and in transit using customer-managed encryption keys (CMEK).

Answer

Remove all direct identifiers such as name and patient ID from the dataset before training.

Answer

Restrict access to the deployed endpoint using IAM and a private VPC endpoint.

Question 30

A retail company runs a 340M-parameter BERT-based text classifier on a Vertex AI endpoint to categorize incoming customer chat messages in real time. The model meets accuracy targets, but p99 latency is 480 ms — well above the 120 ms SLA — and GPU serving costs are high. The team wants to dramatically reduce latency and cost while keeping accuracy within about 2% of the current model, and they are willing to invest in a one-time offline effort. Which approach best meets these requirements?

Accepted Answer

Apply knowledge distillation to train a much smaller student model using the current model as the teacher, then serve the student model.

Answer

Switch the endpoint from online inference to batch inference to process messages more efficiently.

Answer

Increase the minimum replica count and add more GPUs to the endpoint's autoscaling configuration.

Answer

Enable request-response logging and raise the endpoint's maximum concurrent request limit.

Question 31

Your team serves a large BERT-based text classifier on a Vertex AI endpoint with GPU accelerators. Traffic has grown, and GPU serving costs are becoming unsustainable. The business requires that classification accuracy remain within 1% of the current model, and that p95 latency stay under 50 ms. You want to reduce serving cost the most while meeting both constraints, and you have access to the full labeled training dataset plus abundant unlabeled production text. Which optimization approach should you pursue first?

Accepted Answer

Train a much smaller student model via knowledge distillation using the large model as the teacher, then evaluate it against the accuracy and latency constraints on CPU serving.

Answer

Apply post-training int8 quantization to the existing large model and deploy it unchanged on the same GPU endpoint.

Answer

Increase the endpoint's minimum replica count and lower the autoscaling GPU utilization target to spread traffic across more nodes.

Answer

Switch the endpoint from GPU to a larger-memory GPU type to process more requests per replica.

Question 32

Your team has a TensorFlow model that trains on a 4 TB dataset stored in Cloud Storage as TFRecord files. A single-node 8-GPU machine takes over 40 hours per training run, which is blocking your experimentation velocity. The model uses large embedding layers and dense transformer blocks, and the training loop is written with the Keras API. You need to dramatically reduce wall-clock training time while keeping code changes minimal and controlling cost. What is the most appropriate approach?

Accepted Answer

Refactor the training code to use tf.distribute.TPUStrategy and run on a Cloud TPU v-series Pod slice, using tf.data to stream the TFRecords from Cloud Storage.

Answer

Keep the single-node setup but switch the machine type to one with more vCPUs and add more Cloud Storage read parallelism to speed up data loading.

Answer

Migrate the workload to BigQuery ML so the training runs directly where a copy of the data can be loaded, avoiding GPU costs entirely.

Answer

Convert the model to scikit-learn and use joblib parallel backends across multiple CPU nodes to distribute the training.

Question 33

Your team has a TensorFlow 2 image classification model that currently trains on a single GPU and takes 18 hours per run. You provisioned a Vertex AI custom training job on one machine equipped with 4 NVIDIA A100 GPUs and want to reduce training time with minimal code changes while keeping the model architecture unchanged. Which approach should you use to distribute training across the 4 GPUs?

Accepted Answer

Wrap the model construction and compilation inside tf.distribute.MirroredStrategy scope and let it synchronize gradients across the 4 GPUs

Answer

Use tf.distribute.MultiWorkerMirroredStrategy and configure a TF_CONFIG environment variable listing the 4 GPUs as separate workers

Answer

Use tf.distribute.ParameterServerStrategy with one GPU acting as the parameter server and the remaining three as workers

Answer

Manually split each training batch into four shards and run four independent training processes, then average the resulting model weights after each epoch

Question 34

You run a TensorFlow image-classification model on a Vertex AI online endpoint backed by GPUs. Traffic is bursty, and during peak periods GPU utilization stays low (around 30%) even though request latency climbs and clients report timeouts. Each individual request contains a single image. You want to increase throughput and improve GPU efficiency without reducing model accuracy or provisioning more replicas. What is the most effective change?

Accepted Answer

Enable server-side request (dynamic) batching so multiple concurrent single-image requests are grouped into one batch before the GPU forward pass.

Answer

Switch the endpoint from GPU to CPU machines to eliminate the GPU scheduling overhead causing the low utilization.

Answer

Convert the deployment to a batch prediction job that reads requests from Cloud Storage and writes results back.

Answer

Increase the autoscaling maximum replica count and lower the target GPU utilization threshold to 20%.

Question 35

You deployed a gradient-boosted model on a Vertex AI endpoint to predict loan default. Overall prediction accuracy and the marginal distributions of each input feature (monitored via training-serving skew and drift detection) have stayed within alert thresholds for three months. However, business stakeholders report that recent predictions no longer match domain intuition, and a manual audit suggests the model is now weighting certain features very differently than at launch. Which monitoring approach would most directly detect this problem?

Accepted Answer

Enable feature attribution drift monitoring so shifts in each feature's contribution to predictions are tracked over time

Answer

Increase the sampling rate of the existing input feature drift monitoring job

Answer

Add prediction output distribution monitoring to detect changes in the score histogram

Answer

Reduce the training-serving skew alert threshold so smaller deviations trigger alerts

Question 36

A media company deploys a recommendation model that ranks articles for users. The model is retrained weekly using logged user clicks as ground-truth labels. Over several months, the team notices that recommendation diversity collapses—users increasingly see only a narrow set of popular topics, and engagement among niche-interest users declines. Data drift and skew monitoring show no anomalies in input feature distributions. What is the most likely root cause of this degradation?

Accepted Answer

A feedback loop: the model's own recommendations shape the click logs used for retraining, so labels are biased toward previously surfaced items, amplifying popularity bias over successive training cycles.

Answer

Training-serving skew caused by different preprocessing code paths between the training pipeline and the online serving container.

Answer

Concept drift in which the true relationship between features and user preferences has shifted independently of the model's behavior.

Answer

Numerical feature drift that the monitoring thresholds are configured too loosely to detect.

Question 37

Your team has a prototype trained in scikit-learn on a small sample of tabular data. The production dataset has 4 billion rows stored in BigQuery, with several high-cardinality categorical features. You must train a gradient-boosted model that scales to the full dataset, supports distributed training across multiple workers, and integrates cleanly with Vertex AI managed training. Which approach best meets these requirements?

Accepted Answer

Rewrite the model using XGBoost with its distributed (Dask or Spark) backend, run it as a Vertex AI custom training job across multiple workers, and read data in shards from BigQuery.

Answer

Continue with scikit-learn's GradientBoostingClassifier on a single high-memory Vertex AI training VM, loading the data into a pandas DataFrame.

Answer

Export all 4 billion rows to a single CSV in Cloud Storage and train scikit-learn with joblib parallelism on one machine with many vCPUs.

Answer

Convert the tabular data to images and train a convolutional neural network on TPUs for maximum throughput.

Question 38

Your team deployed a resume-screening model to rank candidates for engineering roles. During a responsible AI review, you discover that although gender was explicitly removed from the training data, the model still recommends male candidates at a significantly higher rate. Investigation shows the model relies heavily on features like 'years of continuous employment' and membership in certain college sports clubs, which correlate strongly with gender. What is the MOST appropriate action to address this fairness risk?

Accepted Answer

Identify and mitigate the proxy features driving disparate impact, then evaluate the model against fairness metrics such as demographic parity or equalized odds before redeploying.

Answer

Re-add gender as a feature so the model can be explicitly calibrated to produce equal selection rates across groups.

Answer

Increase the training dataset size so the model learns a more general representation and the correlation disappears naturally.

Answer

Lower the decision threshold uniformly for all candidates to increase the number of people recommended and reduce overall disparity.

Question 39

Your team deployed a generative AI assistant on Vertex AI that answers customer questions by drawing on an internal knowledge base. During a responsible-AI review, security raises a concern: the model occasionally echoes back personally identifiable information (PII) such as email addresses and account numbers that appeared in the prompt or retrieved context. Leadership wants an ongoing safeguard that reduces the risk of PII being surfaced in production responses without retraining the model. Which approach best mitigates this privacy risk?

Accepted Answer

Add a post-processing inspection step that runs each model response through Cloud Data Loss Prevention (DLP) to detect and redact PII before returning it to the user, and log detections for auditing.

Answer

Enable Vertex AI Model Monitoring for feature attribution drift so you can detect when input distributions change over time.

Answer

Increase the model's temperature setting so that responses are more varied and less likely to reproduce exact strings from the context.

Answer

Retrain the model with differential privacy so memorized training examples cannot be reconstructed.

Question 40

Your team hosts 12 lightweight image-classification models on Vertex AI, each receiving low but steady traffic (a few requests per second). Each model fits comfortably in about 2 GB of GPU memory. Currently each model runs on its own dedicated node with a full NVIDIA T4 GPU, and GPU utilization per node hovers around 8%. Leadership wants to cut serving costs without materially increasing latency. What is the most effective way to reduce cost here?

Accepted Answer

Deploy all 12 models to a single Vertex AI endpoint as co-hosted models sharing the same GPU-backed DeployedModel resources, consolidating them onto fewer GPU nodes

Answer

Switch every model to CPU-only nodes and enable dynamic batching to compensate for the loss of GPU acceleration

Answer

Keep one GPU per model but enable autoscaling with a minimum of zero replicas so idle models scale to nothing

Answer

Apply INT8 quantization to each model so they run faster and the GPUs finish requests sooner

Question 41

You are training a large image classification CNN on a single Vertex AI custom training node equipped with one NVIDIA A100 (40 GB) GPU. Training crashes with a CUDA out-of-memory error only after you increased the input image resolution from 224x224 to 512x512. The model architecture and dataset are fixed, and you must keep the higher resolution. You want the simplest change that lets training proceed on the same single-GPU node without significantly degrading final model accuracy. Which action should you take first?

Accepted Answer

Reduce the per-step batch size and use gradient accumulation to maintain the effective batch size

Answer

Switch the training node to a machine with 8 A100 GPUs and enable MirroredStrategy

Answer

Downsample the input images back to 224x224 to fit within GPU memory

Answer

Convert the model to run on a TPU v3 Pod slice to avoid GPU memory limits

Question 42

Your team is training a large vision transformer on a single NVIDIA A100 (40 GB) GPU using PyTorch on Vertex AI. Research shows the model converges best with an effective batch size of 512, but the GPU runs out of memory at a batch size of 64. You must not change hardware and want to preserve the target effective batch size without degrading convergence. What is the most appropriate technique?

Accepted Answer

Enable gradient accumulation over 8 micro-batches of size 64 before applying an optimizer step

Answer

Reduce the batch size to 64 and proportionally lower the learning rate to compensate

Answer

Switch to a MultiWorkerMirroredStrategy across additional GPUs to split the batch

Answer

Cast all model weights to int8 quantization to reduce memory footprint during training

Question 43

Your team serves a TensorFlow image-classification model on a Vertex AI endpoint. A high-volume internal microservice sends millions of prediction requests per hour and is experiencing significant per-request serialization overhead and connection setup latency. The payloads are large tensor arrays. You want to reduce serving latency and increase throughput between the client and the model server without changing the model itself. Which change should you prioritize?

Accepted Answer

Switch the client-to-server communication to use the gRPC protocol with a persistent connection and binary protobuf payloads.

Answer

Increase the endpoint's minimum replica count so more machines are always available to handle load.

Answer

Convert all requests to base64-encoded JSON strings sent over the existing REST/HTTP interface.

Answer

Enable request-response logging on the endpoint to capture and analyze payload sizes.

Question 44

A financial services company serves a large transformer-based text classification model on Vertex AI endpoints backed by NVIDIA T4 GPUs. Traffic has tripled, and p99 latency now regularly exceeds the 150 ms SLA. The team wants to increase throughput per GPU without adding more replicas or retraining the model, and they can tolerate a very small drop in accuracy. Which optimization should they apply first?

Accepted Answer

Apply post-training INT8 quantization to the model and validate accuracy on a holdout set before redeploying.

Answer

Convert the model to FP64 precision to improve numerical stability during inference.

Answer

Enable knowledge distillation at serving time so the endpoint compresses requests dynamically.

Answer

Increase the endpoint's minReplicaCount so more GPUs share the load.

Question 45

You maintain a Vertex AI Pipeline built with the Kubeflow Pipelines SDK. It contains several lightweight Python function-based components that each import pandas and scikit-learn. Recently, a pipeline run failed because one component silently pulled a newer scikit-learn version than the one used during development, causing an incompatible model artifact. You want to guarantee that every future run of these components executes with the exact same library versions, while keeping component authoring simple and reproducible. What is the best approach?

Accepted Answer

Build a custom container image with the pinned library versions, push it to Artifact Registry, and reference it as the base_image (and packages_to_install where needed) for each function-based component.

Answer

Add a pip install command with unpinned package names inside each component function so dependencies are freshly resolved at runtime.

Answer

Enable pipeline component caching so the previously successful execution is always reused instead of re-running the component.

Answer

Increase the machine type and memory for the affected component so the newer library version has enough resources to run correctly.

Question 46

Your team builds a Vertex AI Pipeline (KFP v2) that must train the same model architecture on 12 different regional datasets. Each regional training run is independent, and the datasets are known at compile time from a list. Downstream, a single aggregation component must collect the evaluation metrics from all 12 runs to select the best regional model. You want to express this fan-out/fan-in pattern efficiently in the pipeline DSL without hardcoding 12 separate component calls. Which approach should you use?

Accepted Answer

Use a dsl.ParallelFor loop over the list of regions to fan out the training component, then use dsl.Collected to gather the outputs into the aggregation component.

Answer

Compile 12 separate pipelines and trigger them concurrently, then run a manual script afterward to compare the metrics files in Cloud Storage.

Answer

Write a single training component that internally loops over all 12 regions sequentially, then pass its combined output to the aggregation component.

Answer

Use a dsl.Condition block for each region and rely on component caching to reuse results across the branches.

Question 47

Your team runs a Vertex AI Pipeline (built with the KFP SDK) that trains a model and produces a metrics artifact for each run. Product managers want a self-service way to compare the accuracy and AUC of models across many pipeline runs over time, without manually inspecting logs. You want to use built-in capabilities with minimal custom code. What should you do?

Accepted Answer

Have the training component write metrics into a Metrics artifact using log_metric() on the output artifact, then use the Vertex AI ML Metadata / Pipelines run comparison UI to compare runs.

Answer

Print the metrics to stdout in the training component and instruct product managers to grep the Cloud Logging entries for each pipeline run.

Answer

Write each run's metrics to a separate text file in a Cloud Storage bucket and share the bucket so managers can open individual files.

Answer

Store metrics only inside the trained model's saved artifact and require managers to load each model to read them.

Question 48

Your team runs a Vertex AI Pipeline (compiled from KFP) that includes a component calling an external REST API to enrich features. Intermittently this component fails due to transient network timeouts, causing the entire pipeline run to fail hours into execution. You want the pipeline to automatically recover from these transient failures without re-executing already-completed upstream components. What is the most appropriate approach?

Accepted Answer

Configure a retry policy on the enrichment component using the task's set_retry() method with a backoff, and ensure upstream component outputs are cached so they are not recomputed.

Answer

Wrap the entire pipeline in an ExitHandler that re-submits the whole pipeline run whenever any component fails.

Answer

Increase the machine size for the enrichment component so the API calls complete faster and avoid timeouts.

Answer

Disable caching globally so that on restart every component runs again from a clean state, guaranteeing consistency.

Question 49

Your team is building a Vertex AI Pipeline using the Kubeflow Pipelines SDK v2. A preprocessing component produces a large transformed dataset (several GB) that must be consumed by a downstream training component. You want the pipeline to pass this data between components in a way that supports lineage tracking, caching, and avoids embedding the data directly in the pipeline's parameter payload. How should you define the data flow between the two components?

Accepted Answer

Define the preprocessing component with an Output[Dataset] artifact and have the training component accept an Input[Dataset] artifact, letting the SDK handle GCS staging and metadata.

Answer

Have the preprocessing component return the dataset as a string parameter output and pass it as an input parameter to the training component.

Answer

Write the dataset to a hardcoded GCS bucket path inside the preprocessing component and read from the same hardcoded path in the training component.

Answer

Store the dataset in a global Python variable in the preprocessing component so the training component can reference it directly at runtime.

Question 50

You are building a Vertex AI Pipeline using the Kubeflow Pipelines (KFP) v2 SDK. The pipeline has three components: a lightweight data validation step, a CPU-heavy feature engineering step, and a GPU-intensive model training step. During test runs, all steps are provisioned with the same default machine type, causing the training step to run out of memory while the validation step wastes expensive resources. You want each component to request appropriate compute resources without changing pipeline logic. What is the recommended approach?

Accepted Answer

Chain resource specification methods such as .set_cpu_limit(), .set_memory_limit(), and .add_node_selector_constraint() (or .set_accelerator_type()) on each component task within the pipeline definition.

Answer

Split the pipeline into three separate pipelines, each compiled and submitted with a different worker pool specification so machine types can vary.

Answer

Set a single global machine type in the PipelineJob submission request that satisfies the GPU training step, since Vertex AI applies it to all components.

Answer

Rewrite the components as custom training jobs and configure machine types through the Vertex AI Training API instead of the KFP SDK.

Question 51

Your team runs a Vertex AI Pipeline (built with the Kubeflow Pipelines SDK v2) that trains a model, uploads artifacts to Cloud Storage, and spins up a temporary Dataproc cluster for a feature-engineering step. Occasionally a downstream component fails partway through the run. When this happens, the temporary Dataproc cluster is left running (accruing cost) and no one is notified of the failure. You need to guarantee that a cleanup-and-notify task always executes at the end of the pipeline, whether the run succeeds or fails. What is the best way to implement this?

Accepted Answer

Wrap the pipeline's main steps inside a dsl.ExitHandler and attach a cleanup-and-notify component as the exit task so it runs regardless of upstream success or failure.

Answer

Add the cleanup-and-notify component as the last step in the pipeline DAG with an explicit .after() dependency on the final training component.

Answer

Configure component-level retries with dsl.RetryPolicy on every task so failures are automatically retried and cleanup never becomes necessary.

Answer

Enable component caching on all tasks so that on a re-run the pipeline skips completed steps and reaches the cleanup step faster.

Question 52

Your ML platform team maintains several Vertex AI Pipelines across different product teams. Multiple pipelines all need an identical data-validation step that your team authored as a Kubeflow (KFP) component. Currently each team copies the component's Python source into their own repository, causing drift and inconsistent behavior when the validation logic is updated. You want teams to consume a single, centrally-maintained, versioned copy of the component with minimal duplication. What should you do?

Accepted Answer

Compile the component to YAML, publish versioned artifacts to Artifact Registry (KFP template repository), and have each team load the component by version reference at build time.

Answer

Enable component caching in every pipeline so the validation step is reused across runs without teams needing to copy the source.

Answer

Package the validation logic as a custom training container image and have each pipeline invoke it through a CustomJob operator.

Answer

Store the component's Python file in a shared Cloud Storage bucket and have each team download and paste it into their DSL at authoring time.

Question 53

Your team has deployed 12 small TensorFlow models to Vertex AI, each on its own dedicated online endpoint with one n1-standard-4 node. Traffic to each model is low and sporadic, but the endpoints must remain available for occasional real-time requests. Finance flags that you are paying for 12 always-on nodes even though average utilization per endpoint is under 5%. You need to significantly reduce serving cost while keeping low-latency online inference available. What should you do?

Accepted Answer

Deploy multiple models to a single Vertex AI endpoint and place them on a shared DeployedModel/node pool via co-hosting (model co-hosting / deployment resource pools), sharing compute across the low-traffic models.

Answer

Switch all 12 models to Vertex AI batch prediction jobs and run them hourly so no nodes stay running between requests.

Answer

Keep the 12 endpoints but reduce each machine type to n1-standard-2 and disable autoscaling to lock in a lower fixed cost.

Answer

Merge the 12 models into one large ensemble model artifact and deploy it to a single endpoint with a GPU accelerator.

Question 54

Your ML platform team runs a Vertex AI Pipeline that trains a model, evaluates it, and pushes it to the Vertex AI Model Registry. Compliance requires that no model version reach the production endpoint until a designated reviewer manually inspects the evaluation metrics and explicitly approves the release. You want to keep the training-through-registration steps fully automated while inserting this human approval before deployment, without rebuilding or retraining the model after approval. What is the best way to design this?

Accepted Answer

Split the workflow: an automated training/evaluation/registration pipeline that stops after registering the candidate version, and a separate deployment pipeline triggered only after a reviewer approves (e.g., via a Model Registry version alias/label update that fires an Eventarc/Pub-Sub trigger).

Answer

Add a KFP dsl.Condition step in the single pipeline that pauses execution and waits for the reviewer to type approval into the pipeline run's console before continuing to deployment.

Answer

Use an exit handler at the end of the pipeline that deploys the model automatically and emails the reviewer so they can roll back if the metrics were unacceptable.

Answer

Have the pipeline deploy the model to the production endpoint immediately, but set the traffic split to 0% until the reviewer manually raises it to 100%.

Question 55

Your team runs a Vertex AI Pipeline daily. A preprocessing component reads a static reference table from Cloud Storage and produces a transformed dataset artifact. You enabled execution caching to avoid recomputing unchanged steps. This week, the reference file in Cloud Storage was updated, but the pipeline continued reusing the cached output from the preprocessing component and never reprocessed the new data. What is the most likely cause?

Accepted Answer

Vertex AI Pipelines caching keys are computed from component inputs (parameters and input artifact metadata) and the component specification, not from the contents of files read internally via a hardcoded Cloud Storage path.

Answer

Execution caching in Vertex AI Pipelines expires only after 24 hours, so the same-day run reused the result before the cache became invalid.

Answer

Caching automatically detects any change to files in the pipeline's project bucket, but only if the bucket has object versioning enabled.

Answer

The pipeline was compiled with caching set at the pipeline level, which ignores per-component input changes entirely.

Professional Machine Learning Engineer (PMLE) full-length practice exam

How this timed PMLE mock exam works

What your PMLE results show

How to get the most out of it

Ready for the full PMLE mock exam?