Prompt Caching on Bedrock

Prompt caching on Amazon Bedrock lets you cache specific portions of your requests for repeated use. This feature significantly reduces inference response latency and input token costs by allowing the model to skip recomputation of previously processed content.

With Portkey, you can easily implement Amazon Bedrock’s prompt caching through our OpenAI-compliant unified API and prompt templates.

​

Model Support

Amazon Bedrock prompt caching is generally available with the following models:

​

How Bedrock Prompt Caching Works

When using prompt caching, you define cache checkpoints - markers that indicate parts of your prompt to cache. These cached sections must be static between requests; any alterations will result in a cache miss.

​

Implementation Examples

Here’s how to implement prompt caching with Portkey:

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    virtualKey: "VIRTUAL_KEY" // Your Bedrock Virtual Key
})

const chatCompletion = await portkey.chat.completions.create({
    messages: [
        { "role": 'system', "content": [
            {
                "type":"text","text":"You are a helpful assistant"
            },
            {
                "type":"text","text":"This is a large document I want to cache...",
                "cache_control": {"type": "ephemeral"}
            }
        ]},
        { "role": 'user', "content": 'Summarize the above document for me in 20 words' }
    ],
    model: 'anthropic.claude-3-7-sonnet-20250219-v1:0'
});

console.log(chatCompletion.choices[0].message.content);

​

Supported Features and Limitations

​

Supported Models and Limits

Below is a detailed table of supported models, their minimum token requirements, maximum cache checkpoints, and fields that support caching:

​

Related Resources