What is Prompt Caching?
Reusing computation for prompt content that an AI service has already processed, often a shared prefix containing instructions, tools, or reference material. On supported models, a cache hit can reduce input processing cost and latency. Eligibility, retention, and billing depend on the provider.
My perspective
In practice
I place stable instructions and shared context before request-specific material, then measure cache hits rather than assuming reuse. Caching does not make the model remember a conversation or guarantee the same output. Sensitive data still needs the provider's retention and isolation controls to match the application's requirements.