Question

Your team maintains a fraud-scoring model deployed on a Vertex AI online endpoint. A new business requirement asks you to re-score the entire historical transaction table (2 billion rows) once, to backfill risk labels for an analytics dashboard. There is no latency requirement for this one-time job, but you want to minimize cost and total wall-clock time. The online endpoint currently autoscales between 3 and 10 GPU replicas. What is the most appropriate approach?

Accepted Answer

Run a Vertex AI batch prediction job that reads the table from BigQuery, using a high machine count to parallelize the workload.

Answer

Send the 2 billion rows to the existing online endpoint by fanning out many concurrent HTTP requests and temporarily raising max replicas to 50.

Answer

Deploy a second online endpoint dedicated to the backfill and stream all rows through it with dynamic batching enabled.

Answer

Export the online endpoint model and run inference locally in a single notebook loop over all 2 billion rows.