Prompt Caching
Prompt caching is a feature that stores a reusable prefix of a prompt so repeated requests reuse it, reducing latency and cost. It is valuable when a large, stable context (such as a document or system prompt) is sent on many requests.