Question

A company runs a customer-facing legal document summarization assistant on Amazon Bedrock. Each request sends a large document and asks the model to produce a 600-800 word summary. Users complain that the interface appears frozen for 10-15 seconds after they submit a request, even though the total generation time is acceptable. The engineering team wants to improve the perceived responsiveness without changing the model, reducing output length, or degrading summary quality. Which approach best addresses the complaint?

Accepted Answer

Switch from the InvokeModel API to the InvokeModelWithResponseStream API and render tokens in the UI as they arrive.

Answer

Purchase Provisioned Throughput for the model to guarantee dedicated capacity and lower total generation time.

Answer

Enable prompt caching on the document context so repeated requests return faster.

Answer

Reduce the max_tokens parameter to cap the output and return the response sooner.