Question

A retail company runs a large, highly capable Gemini model to power its product recommendation explanations, but inference costs are becoming unsustainable at their transaction volume. They want a smaller model that mimics the behavior and quality of the large model for this specific task, so they can deploy it at lower cost and latency. Which Vertex AI capability best addresses this need?

Accepted Answer

Model distillation, which trains a smaller student model to replicate the outputs of the larger teacher model

Answer

Grounding with Google Search to reduce the model's reliance on its parameters

Answer

Deploying additional TPU pods to horizontally scale the existing large model

Answer

Retrieval-augmented generation (RAG) to fetch recommendations from a vector database