Python: .NET: Python: Native support for Bedrock-hosted models (Anthropic, Cohere, etc.) in Python SDK #2610
Conversation
…g the review comments
markwallace-microsoft
added
documentation
github-actions
bot
changed the title
|
@eavanvalkenburg - Could you review the latest PR? |
|
@duttastunil please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
|
@eavanvalkenburg - Could you please review my PR? Thanks!! |
eavanvalkenburg
left a comment
eavanvalkenburg
left a comment
There was a problem hiding this comment.
Sorry, I missed a couple of things in earlier looks, especially around parsing content to and from. we also changed our approach to new packages slightly so some stuff needs to be (re)moved. Finally, you need to add this package to the project packages list in the uv section of the root level pyproject.toml
| class BedrockChatClient(BaseChatClient): | ||
| """Async chat client for Amazon Bedrock's Converse API.""" | ||
|
|
||
| OTEL_PROVIDER_NAME: ClassVar[str] = "bedrock" # type: ignore[reportIncompatibleVariableOverride, misc] |
There was a problem hiding this comment.
just looked up this one, they want it to be aws.bedrock: https://opentelemetry.io/docs/specs/semconv/gen-ai/aws-bedrock/
| return blocks or [{"text": ""}] | ||
| return [self._normalize_tool_result_value(result)] | ||
|
|
||
| def _normalize_tool_result_value(self, value: Any) -> dict[str, Any]: |
There was a problem hiding this comment.
we also have a function called prepare_function_call_results (in _types, but exposed through agent-_framework)you might want to use that instead of this, one to properly catch all variants, and make it consistent across the board.
| pending_tool_use_ids: deque[str] = deque() | ||
| for message in messages: | ||
| if message.role == Role.SYSTEM: | ||
| text_value = self._gather_text_from_message(message) |
There was a problem hiding this comment.
this can just use message.text that already appends all texts from text contents within the message.
| return blocks | ||
|
|
||
| def _convert_content_to_bedrock_block(self, content: Any) -> dict[str, Any] | None: | ||
| if isinstance(content, TextContent) and getattr(content, "text", None): |
There was a problem hiding this comment.
no need to safely check, TextContent has a text field
| if isinstance(content, TextContent) and getattr(content, "text", None): | |
| if isinstance(content, TextContent): |
| blocks.append({"text": text_value}) | ||
| return blocks | ||
|
|
||
| def _convert_content_to_bedrock_block(self, content: Any) -> dict[str, Any] | None: |
There was a problem hiding this comment.
with from agent_framework import Contents
| def _convert_content_to_bedrock_block(self, content: Any) -> dict[str, Any] | None: | |
| def _convert_content_to_bedrock_block(self, content: Contents) -> dict[str, Any] | None: |
| block = self._convert_content_to_bedrock_block(content) | ||
| if block: | ||
| blocks.append(block) | ||
| if not blocks: |
There was a problem hiding this comment.
This is not right, (almost) all messages have contents, so those should be parsed by the above block. This fallback should not be hit, then the _convert_content_to_bedrock_block should parse all contents in a appropriate way. Also the text attribute is present on both TextContent and TextReasoningContent, and those should not be mixed. So parse contents properly (a ChatMessage created with a text param, get's parsed to a TextContent in the contents) and decide what happens when a message contains contents that are incompatible with the api (ignore the content itself, but what if the message is then empty, skip?)
There was a problem hiding this comment.
I'm sorry, we decided that for the initial launch of a new package, we want to not depend on it, here, so could you remove these again? Once we have done the first release and everything works we can add them again.
There was a problem hiding this comment.
and then this sample will also temporarily move into the bedrock folder, and back to here once we are confident about everything working well.
| ai_function, | ||
| ) | ||
|
|
||
| from agent_framework.amazon import BedrockChatClient |
There was a problem hiding this comment.
This will also be updated once the import is recreated and stable. So for now:
| from agent_framework.amazon import BedrockChatClient | |
| from agent_framework_bedrock import BedrockChatClient |
3 participants
Motivation and Context
The current Python SDK for the Microsoft Agent Framework requires users to route requests to Bedrock models via the generic OpenAI-compatible API interface. This approach has several drawbacks:
It does not natively recognize specific model IDs (e.g., anthropic.claude-3-5-sonnet-20240620-v1:0).
It may lack native support for provider-specific parameters or functionality (like Anthropic's specific messaging API structure, context windows, or token cost tracking).
It creates a less intuitive configuration experience compared to native integrations for Azure OpenAI or generic OpenAI services.
I would like to request dedicated, native client classes and configuration support within the Python SDK to directly integrate models hosted on Amazon Bedrock.
Description
Below are the details of the BedrockChatClient:
Async implementation of the Agent Framework BaseChatClient that targets Amazon Bedrock’s Converse API.
Handles provider setup: reads Bedrock credentials/region from env (via BedrockSettings), builds a boto3 bedrock-runtime client with the AF user-agent, and exposes service_url.
Converts framework messages and tools into Bedrock-compatible payloads (system prompts, content blocks, tool specs/choices) while enforcing Bedrock request rules (e.g., aligning tool results with pending tool calls).
Issues converse calls and adapts results back into Agent Framework structures: ChatResponse, FunctionCallContent, FunctionResultContent, usage metrics, and finish reasons.
Supports streaming via _inner_get_streaming_response, automatically adds tool usage telemetry (use_function_invocation), middleware hooks, and OpenTelemetry metadata (use_observability).
Contribution Checklist