Question

A legal-tech company runs a document-review assistant on Amazon Bedrock. Each user query re-sends the same lengthy 40-page policy manual as context, followed by a short unique question. The team observes high per-request costs and slow response times because the large manual is re-processed every time. They want to reduce both cost and latency without removing the manual from the context or changing which foundation model they use. Which approach best addresses this?

Accepted Answer

Enable prompt caching so the repeated manual context is stored and reused across requests instead of being reprocessed each time

Answer

Increase the temperature parameter to speed up token generation

Answer

Fine-tune the foundation model on the policy manual so the context no longer needs to be sent

Answer

Raise the max tokens limit so the model returns answers faster