Question

A customer support SaaS company routes all incoming tickets through a single large foundation model on Amazon Bedrock. Analysis shows that roughly 70% of tickets are simple FAQ-style questions that a smaller, cheaper model answers correctly, while 30% are complex, multi-issue cases that require the large model's reasoning quality. The team wants to cut inference costs significantly without degrading answer quality on the complex cases. Which approach BEST achieves this goal?

Accepted Answer

Implement a lightweight classifier that scores each ticket's complexity and routes simple tickets to a smaller model while sending complex tickets to the large model.

Answer

Move all traffic to batch inference so both simple and complex tickets are processed at the lower batch pricing tier.

Answer

Reduce the max_tokens parameter on the large model for all tickets to lower per-request token consumption.

Answer

Purchase provisioned throughput for the large model to obtain a discounted rate for the full ticket volume.