Question

A retail company is building a customer-facing product-recommendation agent in Microsoft Foundry. Traffic is highly variable: roughly 70% of incoming prompts are simple lookups (e.g., 'What is my order status?') that need fast, low-cost responses, while 30% are complex, multi-turn styling requests that require deep reasoning and higher-quality output. The architect wants to minimize cost without degrading the experience for complex requests, and wants the routing decision to be handled automatically rather than by hard-coded application logic. Which approach should the architect implement?

Accepted Answer

Deploy a model router that evaluates each incoming request and dynamically dispatches simple prompts to a smaller, cheaper model and complex prompts to a larger reasoning model.

Answer

Deploy a single large reasoning model for all requests to guarantee consistent quality across simple and complex prompts.

Answer

Deploy a single small language model for all requests to guarantee the lowest possible per-token cost across the workload.

Answer

Implement application-level if/else logic that inspects prompt length and manually calls the appropriate model endpoint.