Question

You are building a customer-support assistant that answers questions about a 40-page product manual. Every request sends the same large manual as context in the system prompt, followed by a different short user question. Latency and cost are climbing as traffic grows. Which Messages API feature best addresses this while keeping the manual available on every turn?

Accepted Answer

Enable prompt caching on the large, unchanging manual portion so it is reused across requests

Answer

Lower the temperature parameter to reduce the number of tokens processed

Answer

Set a stop sequence after the manual so the model ignores it during generation

Answer

Reduce max_tokens so the model reads less of the input context