Question

A financial services company is designing a real-time fraud detection service on Google Cloud. The model is a deep neural network that requires GPU acceleration to meet a strict 50 ms p99 inference latency target. Traffic is highly variable: near-zero overnight and peaking at 30,000 requests per second during business hours. The architecture team must minimize cost while meeting the latency SLA and avoiding manual capacity management. Which compute design should the architect recommend?

Accepted Answer

Deploy the model to Vertex AI online prediction endpoints with GPU-backed nodes and configure autoscaling with a minimum and maximum node count tied to utilization.

Answer

Deploy the model on a fixed fleet of Compute Engine instances with attached GPUs, sized for peak traffic and left running 24/7 for predictable performance.

Answer

Run the model as a Cloud Run service without GPUs, relying on high concurrency and horizontal scaling to absorb peak load.

Answer

Batch incoming requests and process them on a Dataproc cluster with GPU workers scheduled to run during business hours.