Question

A retail company runs a customer-facing chatbot on a large foundation model in Amazon Bedrock. Users complain that responses are slow, and the finance team reports that per-request costs are high. Investigation shows the model is generating long, verbose answers averaging 800 tokens each, even for simple questions, and the app must display responses within 2 seconds. Which change would MOST directly reduce both latency and cost without switching to a different foundation model?

Accepted Answer

Add instructions in the prompt to answer concisely and lower the max token limit for generated output.

Answer

Increase the temperature parameter so the model produces more creative and varied responses.

Answer

Implement a Retrieval Augmented Generation pipeline to attach more context to each prompt.

Answer

Fine-tune the foundation model on the company's full product catalog.