Question

A legal-tech company runs a document Q&A service on Amazon Bedrock. Each user query prepends the same 8,000-token corporate policy manual as system context, followed by a short user question (roughly 200 tokens). Traffic is bursty and unpredictable throughout the day, averaging about 4,000 requests per hour but occasionally spiking. The team's biggest concern is the high input-token cost driven by the repeated manual, and they want a solution that reduces cost with minimal architecture changes and without committing to a fixed capacity term. Which approach best addresses this?

Accepted Answer

Enable prompt caching on Bedrock so the repeated policy-manual context is cached across invocations, charging the large static prefix at a reduced cached-token rate.

Answer

Purchase provisioned throughput with a 1-month commitment sized to the average request rate to lower per-token costs.

Answer

Switch the workload to Bedrock batch inference to process the queries at the 50% batch discount.

Answer

Reduce the max_tokens output setting to lower the total token bill for each request.