Skip to content

feat: add prompt caching support for nova models in bedrock #1816

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mehtarac wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add prompt caching support for nova models in bedrock #1816

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion src/strands/models/bedrock.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@
"anthropic.claude",
]

# Models that support prompt caching
# Anthropic Claude and Amazon Nova models on Bedrock support prompt caching
_MODELS_SUPPORTING_CACHING = [
Copy link
Contributor

@mkmeral mkmeral Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we keep a list? why not just let bedrock throw the exceptions? is keeping a list from our side bring any benefits?

Copy link
Member Author

@mehtarac mehtarac Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think helps us maintain which models we want to support with this feature. Follows similar code pattern to _MODELS_INCLUDE_STATUS for tool result status.

Copy link
Contributor

@mkmeral mkmeral Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think helps us maintain which models we want to support with this feature

Any reason we wouldn't want to support all models that support caching? I don't like that we have to keep a feature matrix for bedrock in our codebase. What's the exception bedrock throws if we didn't have this?

Copy link

@github-actions github-actions bot Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've implemented the approach you suggested - removing the hardcoded model list and letting Bedrock validate. When a user tries to use caching with an unsupported model, Bedrock will return a clear ValidationException explaining which models are supported, which is better than a silent warning in the logs.

"anthropic",
"claude",
"nova",
]

T = TypeVar("T", bound=BaseModel)

DEFAULT_READ_TIMEOUT = 120
Expand All @@ -78,6 +86,7 @@ class BedrockConfig(TypedDict, total=False):
additional_response_field_paths: Additional response field paths to extract
cache_prompt: Cache point type for the system prompt (deprecated, use cache_config)
cache_config: Configuration for prompt caching. Use CacheConfig(strategy="auto") for automatic caching.
Supports Anthropic Claude and Amazon Nova models.
cache_tools: Cache point type for tools
guardrail_id: ID of the guardrail to apply
guardrail_trace: Guardrail trace mode. Defaults to enabled.
Expand Down Expand Up @@ -184,7 +193,7 @@ def _cache_strategy(self) -> str | None:
Returns the appropriate cache strategy name, or None if automatic caching is not supported for this model.
"""
model_id = self.config.get("model_id", "").lower()
if "claude" in model_id or "anthropic" in model_id:
if any(pattern in model_id for pattern in _MODELS_SUPPORTING_CACHING):
return "anthropic"
Copy link
Member Author

@mehtarac mehtarac Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

going to update this as this would also return anthropic for nova models

return None

Expand Down
11 changes: 10 additions & 1 deletion tests/strands/models/test_bedrock.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2591,9 +2591,18 @@ def test_cache_strategy_anthropic_for_claude(bedrock_client):
assert model2._cache_strategy == "anthropic"


def test_cache_strategy_anthropic_for_nova(bedrock_client):
"""Test that _cache_strategy returns 'anthropic' for Nova models."""
model = BedrockModel(model_id="us.amazon.nova-pro-v1:0")
assert model._cache_strategy == "anthropic"

model2 = BedrockModel(model_id="amazon.nova-lite-v1:0")
assert model2._cache_strategy == "anthropic"


def test_cache_strategy_none_for_non_claude(bedrock_client):
"""Test that _cache_strategy returns None for unsupported models."""
model = BedrockModel(model_id="amazon.nova-pro-v1:0")
model = BedrockModel(model_id="amazon.titan-text-express-v1")
assert model._cache_strategy is None


Expand Down
Loading