Question

A retail analytics team needs to score their entire customer base (about 40 million records) once every night to generate propensity scores for a next-day marketing campaign. The scores are consumed by a downstream BigQuery table the following morning, so end-to-end latency of a few hours is acceptable. The team currently keeps a Vertex AI online endpoint with multiple GPU replicas running 24/7 to handle this workload, but finance has flagged the cost as excessive. What is the most appropriate change to reduce cost while meeting requirements?

Accepted Answer

Switch to a Vertex AI batch prediction job that reads from BigQuery, runs on demand each night, and provisions compute only for the duration of the job.

Answer

Keep the online endpoint but reduce it to a single CPU replica and disable autoscaling to lower hourly cost.

Answer

Keep the online endpoint but enable dynamic request batching to increase throughput per replica.

Answer

Deploy the model to a private endpoint inside the VPC to reduce network egress charges during scoring.