Question

You are designing an Azure AI Foundry solution for a retail chain. A customer-service agent must run on in-store kiosk hardware with limited GPU memory and intermittent connectivity, respond to simple product FAQ prompts in under 200 ms, and keep per-device inference costs minimal. Responses are short, text-only, and drawn from a small fixed FAQ set. Which model selection approach best fits these constraints?

Accepted Answer

Deploy a small language model (SLM) such as a Phi-family model that can run locally on the kiosk hardware.

Answer

Deploy a large frontier LLM like GPT-4o to a centralized cloud endpoint and call it from each kiosk.

Answer

Deploy a multimodal vision-language model to each kiosk to maximize future flexibility.

Answer

Deploy an embedding model to each kiosk and perform vector search only, without a generative model.