Explicit Prompt Caching for OpenAI GPT-5.6 on Amazon Bedrock: Architecture and Patterns

Teaser: OpenAI GPT-5.6 Sol, Terra, and Luna now support explicit prompt caching on Amazon Bedrock — mark exactly which prefix to cache, set your TTL, and cut inference costs by up to 70%. This article explains the architecture, caching mechanics, and the three prompt patterns that maximise cache hit rates in production.

Introduction

In July 2026, AWS announced that OpenAI's GPT-5.6 model family — Sol, Terra, and Luna — is now available on Amazon Bedrock with explicit prompt caching. This is not the automatic, opaque caching that some providers offer. It is a developer-controlled cache where you explicitly mark which prefix of your prompt should be cached, for how long, and under what cache key.

The result: up to 70% cost reduction on cached tokens and 50% latency improvement for cache-hit requests. For teams running high-volume LLM workloads — chatbots with long system prompts, RAG pipelines with repeated context prefixes, batch processing with shared instructions — the economics are transformative.

This article explains how explicit prompt caching works on Bedrock, how GPT-5.6's architecture (Sol, Terra, Luna) relates to caching strategy, and the practical patterns for maximising cache efficiency.

GPT-5.6 Model Family

The GPT-5.6 generation introduces three models with distinct caching profiles:

Model	Parameter Count	Context Window	Primary Use Case	Cache Benefit
Sol	~3.5T (MoE)	256K tokens	Complex reasoning, code generation, research	High — long system prompts benefit most
Terra	~1.2T (MoE)	128K tokens	General purpose, enterprise workflows	Medium — balanced cost/speed
Luna	~400B (Dense)	64K tokens	High-throughput, low-latency chat	Lower — shorter prompts, less cacheable surface

The key architectural detail is that Sol and Terra use mixture-of-experts (MoE) architectures with shared cross-attention layers — which means cached KV cache states from the prefix are reusable across different expert activations in the suffix, making prompt caching particularly effective for these models.

How Explicit Prompt Caching Works on Bedrock

Unlike automatic caching (where the provider decides what to cache), explicit prompt caching gives you control via a cachePoint marker in the message sequence:

import boto3

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

response = bedrock.converse(
    modelId="openai.gpt-5.6-sol",
    messages=[
        # System prompt — marked as cacheable
        {
            "role": "system",
            "content": [
                {
                    "text": LONG_SYSTEM_PROMPT,
                    "cachePoint": {"type": "default"}
                }
            ]
        },
        # User message
        {
            "role": "user",
            "content": [{"text": user_query}]
        }
    ],
    inferenceConfig={
        "cacheConfig": {
            "enabled": True,
            "ttlSeconds": 300  # Cache lives 5 minutes
        }
    }
)

Cache Mechanics

The explicit cache follows a straightforward lifecycle:

sequenceDiagram
    participant App as Application
    participant Bedrock as Bedrock API
    participant Cache as KV Cache Store
    participant Model as GPT-5.6 Sol

    App->>Bedrock: converse() with cachePoint marker
    Bedrock->>Cache: Lookup cache key (prompt prefix hash)
    alt Cache Hit
        Cache-->>Bedrock: Cached KV states
        Bedrock->>Model: Forward only suffix tokens
        Model-->>Bedrock: Generated output
        Bedrock-->>App: Response (cache hit, 50% faster)
    else Cache Miss
        Cache-->>Bedrock: No cached entry
        Bedrock->>Model: Full forward pass
        Model-->>Bedrock: Generated output + KV cache
        Bedrock->>Cache: Store KV cache (TTL: 300s)
        Bedrock-->>App: Response (cache miss, full cost)
    end

1. Cache point — You insert a cachePoint marker before the portion of the prompt you expect to repeat. The KV cache from the start of the prompt up to the cache point is stored.

2. Cache key — The cache is keyed by the exact text prefix up to the cache point. Any change — even a single byte — invalidates the cache entry for that key.

3. TTL — Cache entries expire after the configured ttlSeconds (range: 60–3600). Each cache hit resets the TTL.

4. Cache busting — Bypass the cache by omitting the cachePoint marker or setting enabled: false.

Cost Impact

Bedrock charges per model per token, with cached tokens priced significantly lower:

Model	Input tokens (per 1K)	Cached input tokens (per 1K)	Savings
GPT-5.6 Sol	$0.015	$0.0045	70%
GPT-5.6 Terra	$0.005	$0.0015	70%
GPT-5.6 Luna	$0.0015	$0.0006	60%

Output token pricing is unchanged. The savings apply only to input tokens before the cache point.

Architectural Patterns for Cache Maximisation

Pattern 1: Static System Prompt with Versioned Cache

The most straightforward pattern caches the system prompt, which by definition repeats across all conversations in the same application:

SYSTEM_PROMPT_VERSION = "v2.3"  # Bump when system prompt changes

def build_messages(user_query: str, context: dict) -> list:
    system = compile_system_prompt(context)
    # Include version in preamble to avoid stale cache
    return [
        {
            "role": "system",
            "content": [
                {
                    "text": f"[SYSTEM v{SYSTEM_PROMPT_VERSION}]\n{system}",
                    "cachePoint": {"type": "default"}
                }
            ]
        },
        {
            "role": "user",
            "content": [{"text": user_query}]
        }
    ]

Cache hit rate for stable production prompts: 85–95%, depending on prompt length and TTL.

Pattern 2: Prefix RAG Context

In RAG pipelines, the retrieved documents change per query, but the instruction prefix (how to use context, output format rules) is static. Place the cachePoint between the static instruction and the dynamic context:

def build_rag_prompt(query: str, documents: list[str]) -> list:
    return [
        {
            "role": "system",
            "content": [
                {
                    "text": RAG_INSTRUCTION,  # Static: how to use context
                    "cachePoint": {"type": "default"}  # ← Cache here
                },
                {
                    "text": format_documents(documents)  # Dynamic: per-query
                },
                {
                    "text": f"\n\nQuestion: {query}"
                }
            ]
        }
    ]

This is more effective than caching the entire system prompt because a typical RAG instruction is 500–2000 tokens (highly cacheable), while the document context changes every call. Cache hit rate: 50–70% on the instruction prefix.

Pattern 3: Session-Level Cache for Conversations

In multi-turn chat, the conversation history grows with each turn. Cache the earlier turns that remain static:

def build_chat_messages(conversation_history: list, new_query: str) -> list:
    # Only cache up to the last user-assistant pair
    if len(conversation_history) >= 2:
        cached_prefix = conversation_history[:-2]  # All except last exchange
        new_messages = conversation_history[-2:] + [
            {"role": "user", "content": [{"text": new_query}]}
        ]
    else:
        cached_prefix = conversation_history
        new_messages = [{"role": "user", "content": [{"text": new_query}]}]

    return [
        *cached_prefix,
        {  # Cache point marks where prefix ends
            "role": "assistant",
            "content": [{"text": "", "cachePoint": {"type": "default"}}]
        },
        *new_messages
    ]

Cache hit rate after 3+ turns: 60–75%, scaling with conversation length.

Benchmarks

AWS published reference benchmarks for the caching performance on GPT-5.6 Sol:

Workload	Prompt Size	Cache Hit Rate	Latency (hit)	Latency (miss)	Cost/Request (hit)	Cost/Request (miss)
Chat (long system prompt)	8K tokens	92%	480ms	1,120ms	$0.036	$0.12
RAG pipeline	4K instruction + 12K docs	68% (instruction)	720ms	1,450ms	$0.069	$0.24
Code review (per-file)	6K system + 2K code	88%	510ms	1,080ms	$0.036	$0.12
Batch classification	1.5K instruction	96%	210ms	620ms	$0.006	$0.02

The latency improvement on cache hits is primarily from skipping the KV cache recomputation for attention layers — the dominant cost in transformer inference.

Cache Strategy for Multi-Model Workflows

If you orchestrate across multiple GPT-5.6 models (e.g., Sol for reasoning, Luna for classification), design cache keys to be model-agnostic where possible:

CACHE_CONFIGS = {
    "sol": {"ttlSeconds": 300, "enabled": True},
    "terra": {"ttlSeconds": 600, "enabled": True},  # Longer TTL for stable prompts
    "luna": {"ttlSeconds": 60, "enabled": True},    # Short TTL for fast-changing chat
}

def route_with_cache(query: str, context: dict) -> dict:
    # Determine complexity
    complexity = classify_query(query)

    if complexity == "high":
        model = "openai.gpt-5.6-sol"
    elif complexity == "medium":
        model = "openai.gpt-5.6-terra"
    else:
        model = "openai.gpt-5.6-luna"

    config = CACHE_CONFIGS["sol" if "sol" in model else
                           "terra" if "terra" in model else
                           "luna"]

    return bedrock.converse(
        modelId=model,
        messages=build_messages(query, context),
        inferenceConfig={"cacheConfig": config}
    )

Limitations

Limitation	Details
Region availability	Cache endpoints initially available in us-east-1, us-west-2, and eu-west-1.
Cache size limit	Maximum 256K tokens cached per cache point. Multiple cache points consume separate space.
No cross-model cache	Cache is per-model-id. A cache built for Sol cannot serve Terra requests.
Monitoring gap	CloudWatch metrics for cache hit/miss rates are available but at 1-minute granularity — insufficient for real-time tuning.
TTL precision	TTL is approximate (±10 seconds). High-frequency cache rotation may see miss spikes.

Conclusion

Explicit prompt caching on Bedrock marks a shift in how LLM providers think about caching. By giving developers control over what gets cached, how long it persists, and when it invalidates, AWS and OpenAI make inference optimisation a mainstream engineering concern rather than a vendor-managed black box.

For teams building on GPT-5.6, the immediate takeaway is: design your prompts around the cache point. A well-structured prompt with a static prefix, versioned system instructions, and carefully placed cache markers can deliver 70% cost savings without reducing model quality. In high-volume production, that difference is the line between viable and uneconomical.

Enable explicit prompt caching from the Bedrock Console under Model Configuration → Prompt Caching, or through the cacheConfig parameter in the Converse API.