Question

A legal-tech company runs a document analysis assistant on Amazon Bedrock using Anthropic Claude. Each user session sends the same lengthy 15,000-token contract document as context, followed by dozens of short follow-up questions during the session. The team notices that input token costs dominate their bill because the full document is re-sent and re-processed on every request. They want to reduce cost and time-to-first-token for the repeated document portion without changing the model or splitting the workload into batch jobs. Which approach best addresses this?

Accepted Answer

Enable prompt caching on the Bedrock request, marking the large static document context as a cache checkpoint so repeated tokens are billed at a reduced rate and skip re-processing.

Answer

Purchase Provisioned Throughput for the model to obtain a fixed hourly rate that eliminates per-token charges for the document.

Answer

Switch the workload to Bedrock batch inference to receive the 50% batch discount on the repeated document tokens.

Answer

Reduce the document to a shorter summary using a smaller distilled model before sending it, so fewer input tokens are billed per question.