Question

A legal-tech company runs a document Q&A service on Amazon Bedrock. Each user query is prepended with the same 18,000-token corporate policy handbook plus a 2,500-token system instruction block, followed by a short unique user question (under 200 tokens). Traffic is bursty and unpredictable, with hundreds of queries per hour during business hours and near-zero overnight. The team wants to reduce input-token cost without committing to a fixed hourly spend. Which approach best meets these goals?

Accepted Answer

Enable prompt caching for the static handbook and system instruction prefix, structuring requests so the repeated context is marked as a cache checkpoint while the unique question stays outside the cached segment.

Answer

Purchase Provisioned Throughput with a 1-month commitment sized to peak business-hour demand to guarantee capacity and lower per-token cost.

Answer

Submit all queries through a Bedrock batch inference job every 15 minutes to take advantage of batch pricing discounts on the repeated context.

Answer

Reduce cost by lowering the max_tokens output parameter and switching to a smaller distilled model for all queries.