Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 88 additions & 7 deletions bedrock.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# OpenAI on Amazon Bedrock

The optional `openai-java-bedrock` artifact configures the standard OpenAI Java client for the
OpenAI-compatible Amazon Bedrock Mantle endpoint. It uses the AWS SDK for Java 2.x credential chain
and signs the final HTTP request with SigV4 on every attempt.
OpenAI-compatible Amazon Bedrock Mantle and Runtime endpoints. It uses the AWS SDK for Java 2.x
credential chain and signs the final HTTP request with SigV4 on every attempt. Existing clients
continue to use Mantle by default.

## Installation

Expand All @@ -24,6 +25,66 @@ implementation("com.openai:openai-java-bedrock:4.51.0")

<!-- x-release-please-end -->

## Bedrock Runtime

Select `BedrockEndpoint.RUNTIME` to use the Bedrock Runtime OpenAI-compatible endpoint. This
selects the regional `bedrock-runtime` hostname and the `bedrock` SigV4 signing service while
retaining the standard OpenAI Chat Completions and Responses APIs:

```java
import com.openai.bedrock.BedrockEndpoint;
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.BedrockOpenAIOkHttpClient;
import com.openai.models.chat.completions.ChatCompletionCreateParams;

OpenAIClient client = BedrockOpenAIOkHttpClient.builder()
.endpoint(BedrockEndpoint.RUNTIME)
.awsRegion("us-east-1")
.build();

ChatCompletionCreateParams params = ChatCompletionCreateParams.builder()
.model("us.openai.gpt-5.6-sol")
.addUserMessage("Say hello from Amazon Bedrock Runtime")
.build();

client.chat().completions().create(params).choices().stream()
.flatMap(choice -> choice.message().content().stream())
.forEach(System.out::println);
```

Use the cross-region inference-profile identifier configured for your AWS account and region.
Examples include `us.openai.gpt-5.6-sol`, `us.openai.gpt-5.6-terra`, and
`us.openai.gpt-5.6-luna`; AWS rejects bare model identifiers for these deployments. Availability,
global inference-profile access, supported API routes, authentication methods, streaming behavior,
and model permissions depend on AWS account configuration and the selected model.

The default Runtime URL is `https://bedrock-runtime.{region}.amazonaws.com/openai/v1` in standard
AWS regions. The SDK selects the appropriate DNS suffix automatically for China, European Sovereign
Cloud, and ISO partitions. Canonical Runtime URLs, including FIPS and dual-stack URLs, also infer
Runtime mode when passed to `baseUrl(...)` or `AWS_BEDROCK_BASE_URL`.

Run the included example with your existing AWS credentials:

```shell
AWS_REGION=us-east-1 BEDROCK_MODEL=us.openai.gpt-5.6-sol \
./gradlew :openai-java-example:run -Pexample=BedrockRuntimeChat
```

Set `BEDROCK_STREAM=true` to stream Chat Completions. Set `BEDROCK_AUTH=bearer` and
`AWS_BEARER_TOKEN_BEDROCK` to use a Bedrock bearer token instead of SigV4. Set `AWS_PROFILE` to
select an explicit AWS profile and ensure a stale environment bearer token does not take precedence.

An opt-in live test exercises Sol, Terra, and Luna inference profiles using real AWS credentials:

```shell
BEDROCK_LIVE_TEST=1 AWS_REGION=us-east-1 \
./gradlew :openai-java-bedrock:test --tests '*BedrockRuntimeLiveTest'
```

Set `BEDROCK_LIVE_AUTH=bearer`, `BEDROCK_LIVE_API=responses`, or
`BEDROCK_LIVE_STREAM=true` to choose the authentication mode, API, and streaming behavior. Use
`BEDROCK_LIVE_MODELS` to provide a comma-separated set of account-enabled inference profiles.

## Standard AWS credentials

Configure AWS credentials as you normally would, then provide the region:
Expand Down Expand Up @@ -54,12 +115,14 @@ Base URL resolution follows this order:

1. `baseUrl(...)`
2. `AWS_BEDROCK_BASE_URL`
3. `https://bedrock-mantle.{region}.api.aws/openai/v1`
3. the regional endpoint selected by `endpoint(...)`:
- Mantle: `https://bedrock-mantle.{region}.api.aws/openai/v1`
- Runtime: `https://bedrock-runtime.{region}.amazonaws.com/openai/v1`

The `bedrock-mantle` SigV4 service name is intentional. Bedrock's OpenAI-compatible route is
model-dependent: models such as `openai.gpt-5.5` use `/openai/v1`, while
`openai.gpt-oss-120b` uses `/v1`. The builder defaults to `/openai/v1`; configure the
model's documented route explicitly when it differs:
Mantle requests use the `bedrock-mantle` SigV4 service name. Runtime requests use the `bedrock`
SigV4 service name. Bedrock's OpenAI-compatible route is model-dependent: models such as
`openai.gpt-5.5` use `/openai/v1`, while `openai.gpt-oss-120b` uses `/v1`. Both endpoint families
default to `/openai/v1`; configure the model's documented route explicitly when it differs:

```java
OpenAIClient client = BedrockOpenAIOkHttpClient.builder()
Expand All @@ -68,6 +131,16 @@ OpenAIClient client = BedrockOpenAIOkHttpClient.builder()
.build();
```

Runtime deployments that require the `/v1` route can be configured similarly:

```java
OpenAIClient client = BedrockOpenAIOkHttpClient.builder()
.endpoint(BedrockEndpoint.RUNTIME)
.awsRegion("us-east-1")
.baseUrl("https://bedrock-runtime.us-east-1.amazonaws.com/v1")
.build();
```

## Named profile

```java
Expand Down Expand Up @@ -115,6 +188,10 @@ OpenAIClient client = BedrockOpenAIOkHttpClient.builder()

Explicit bearer and AWS credential modes are mutually exclusive.

If a shell contains a stale `AWS_BEARER_TOKEN_BEDROCK`, default-chain authentication will use that
token instead of signing with SigV4. Unset the variable, or select an explicit AWS profile,
credentials provider, or static credentials to force SigV4.

## Async and streaming responses

The same client configuration supports asynchronous and streaming calls:
Expand All @@ -135,6 +212,10 @@ pass `authenticationExecutor(...)` to use a caller-owned executor instead.

- Do not ship AWS credentials in browser or untrusted client applications.
- Prefer temporary credentials, roles, profiles, and workload identities over long-lived keys.
- Canonical AWS Bedrock URLs require HTTPS. Their endpoint family and region must match the
selected endpoint and configured signing region.
- An explicitly selected endpoint is required when signing requests for a custom proxy or test
server. This prevents an ambiguous hostname from selecting the wrong SigV4 service.
- Do not log access keys, secret keys, session tokens, bearer tokens, or signed authorization
headers. The SDK redacts `Authorization` and `X-Amz-Security-Token` from its HTTP logs.
- OpenAI workload identity federation and AWS Bedrock SigV4 are separate authentication systems.
129 changes: 109 additions & 20 deletions openai-java-bedrock/src/main/kotlin/com/openai/bedrock/BedrockAuth.kt
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,18 @@ import software.amazon.awssdk.identity.spi.AwsCredentialsIdentity
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain

internal const val BEDROCK_SERVICE = "bedrock-mantle"
internal const val ENV_BEARER_TOKEN = "AWS_BEARER_TOKEN_BEDROCK"
internal const val ENV_BASE_URL = "AWS_BEDROCK_BASE_URL"
private val AWS_REGION_PATTERN = Regex("^[a-z]{2,8}(?:-[a-z0-9]+)+-\\d+$")

internal data class BedrockConfiguration(
val endpoint: BedrockEndpoint,
val baseUrl: String,
val authenticator: HttpRequestAuthenticator,
)

internal data class BedrockAuthOptions(
val endpoint: BedrockEndpoint?,
val awsRegion: String?,
val baseUrl: String?,
val apiKey: String?,
Expand Down Expand Up @@ -106,28 +108,46 @@ internal fun BedrockAuthOptions.resolve(
)
}

val environmentBaseUrl = normalizeEnvironment(getenv(ENV_BASE_URL))
val resolvedRegion by lazy {
val configuredRegion by lazy {
normalizedRegion
?: normalizeEnvironment(getenv("AWS_REGION"))
?: normalizeEnvironment(getenv("AWS_DEFAULT_REGION"))
?: regionProvider()?.id()
}
val resolvedBaseUrl =
normalizedBaseUrl
?: environmentBaseUrl
?: resolvedRegion?.let { "https://bedrock-mantle.$it.api.aws/openai/v1" }
?: throw OpenAIException(
"Bedrock requires an AWS region. Pass `awsRegion` to the builder, or set `AWS_REGION` or `AWS_DEFAULT_REGION`."
)
val normalizedResolvedBaseUrl = normalizeBaseUrl(resolvedBaseUrl)
val resolvedRegion by lazy {
(configuredRegion ?: regionProvider()?.id()).also(::validateRegion)
}
val configuredBaseUrl = normalizedBaseUrl ?: normalizeEnvironment(getenv(ENV_BASE_URL))
val normalizedResolvedBaseUrl =
if (configuredBaseUrl != null) {
normalizeBaseUrl(configuredBaseUrl)
} else {
val region =
resolvedRegion
?: throw OpenAIException(
"Bedrock requires an AWS region. Pass `awsRegion` to the builder, or set `AWS_REGION` or `AWS_DEFAULT_REGION`."
)
val selectedEndpoint = endpoint ?: BedrockEndpoint.MANTLE
val hostname =
when (selectedEndpoint) {
BedrockEndpoint.MANTLE -> "bedrock-mantle.$region.api.aws"
BedrockEndpoint.RUNTIME ->
"bedrock-runtime.$region.${runtimeDnsSuffixes(region).first}"
}
"https://$hostname/openai/v1"
}
val parsedEndpoint = parseBedrockEndpointHostname(normalizedResolvedBaseUrl.toHttpUrl().host)
val resolvedEndpoint = endpoint ?: parsedEndpoint?.endpoint ?: BedrockEndpoint.MANTLE
val canonicalRegion =
if (parsedEndpoint != null) normalizedRegion?.also(::validateRegion) else null
validateCanonicalEndpoint(normalizedResolvedBaseUrl, resolvedEndpoint, canonicalRegion)
val resolvedAuthenticationExecutor by lazy {
authenticationExecutor ?: newAuthenticationExecutor()
}
val ownsAuthenticationExecutor = authenticationExecutor == null

if (skipAuth) {
return BedrockConfiguration(
resolvedEndpoint,
normalizedResolvedBaseUrl,
NoAuthAuthenticator(normalizedResolvedBaseUrl),
)
Expand All @@ -149,6 +169,7 @@ internal fun BedrockAuthOptions.resolve(

if (bearerSupplier != null) {
return BedrockConfiguration(
resolvedEndpoint,
normalizedResolvedBaseUrl,
BearerAuthenticator(
normalizedResolvedBaseUrl,
Expand All @@ -164,6 +185,12 @@ internal fun BedrockAuthOptions.resolve(
?: throw OpenAIException(
"Bedrock requires an AWS region. Pass `awsRegion` to the builder, or set `AWS_REGION` or `AWS_DEFAULT_REGION`."
)
validateCanonicalEndpoint(normalizedResolvedBaseUrl, resolvedEndpoint, region)
if (endpoint == null && parsedEndpoint == null) {
throw OpenAIException(
"A custom Bedrock endpoint requires an explicit `endpoint` when using AWS credential authentication."
)
}
val (credentialsProvider, ownsCredentialsProvider) =
when {
staticCredentials != null -> AwsCredentialsProvider { staticCredentials } to false
Expand All @@ -174,9 +201,11 @@ internal fun BedrockAuthOptions.resolve(
}

return BedrockConfiguration(
resolvedEndpoint,
normalizedResolvedBaseUrl,
SigV4Authenticator(
baseUrl = normalizedResolvedBaseUrl,
endpoint = resolvedEndpoint,
region = Region.of(region),
credentialsProvider = credentialsProvider,
ownsCredentialsProvider = ownsCredentialsProvider,
Expand Down Expand Up @@ -238,6 +267,64 @@ private fun normalize(name: String, value: String?): String? {
private fun normalizeEnvironment(value: String?): String? =
value?.trim()?.takeIf { it.isNotEmpty() }

private fun validateRegion(region: String?) {
if (region != null && !AWS_REGION_PATTERN.matches(region)) {
throw OpenAIException(
"The Bedrock AWS region is invalid. Use a standard AWS region such as `us-east-1`."
)
}
}

private fun runtimeDnsSuffixes(region: String): Pair<String, String> =
when {
region.startsWith("cn-") -> "amazonaws.com.cn" to "api.amazonwebservices.com.cn"
region.startsWith("eusc-") -> "amazonaws.eu" to "api.amazonwebservices.eu"
region.startsWith("us-iso-") -> "c2s.ic.gov" to "api.aws.ic.gov"
region.startsWith("us-isob-") -> "sc2s.sgov.gov" to "api.aws.scloud"
region.startsWith("eu-isoe-") -> "cloud.adc-e.uk" to "api.cloud-aws.adc-e.uk"
region.startsWith("us-isof-") -> "csp.hci.ic.gov" to "api.aws.hci.ic.gov"
else -> "amazonaws.com" to "api.aws"
}

private data class CanonicalBedrockEndpoint(val endpoint: BedrockEndpoint, val region: String)

private fun parseBedrockEndpointHostname(hostname: String): CanonicalBedrockEndpoint? {
val parts = hostname.removeSuffix(".").lowercase().split('.')
if (parts.size < 3) return null

val service = parts[0]
val region = parts[1]
val suffix = parts.drop(2).joinToString(".")
if (service == "bedrock-mantle" && suffix == "api.aws") {
return CanonicalBedrockEndpoint(BedrockEndpoint.MANTLE, region)
}
if (service == "bedrock-runtime" || service == "bedrock-runtime-fips") {
val (standardSuffix, dualStackSuffix) = runtimeDnsSuffixes(region)
if (suffix == standardSuffix || suffix == dualStackSuffix) {
return CanonicalBedrockEndpoint(BedrockEndpoint.RUNTIME, region)
}
}
return null
}

private fun validateCanonicalEndpoint(baseUrl: String, endpoint: BedrockEndpoint, region: String?) {
val parsedUrl = baseUrl.toHttpUrl()
val canonical = parseBedrockEndpointHostname(parsedUrl.host) ?: return
if (parsedUrl.scheme != "https") {
throw OpenAIException("Canonical Amazon Bedrock endpoints require HTTPS.")
}
if (canonical.endpoint != endpoint) {
throw OpenAIException(
"The Bedrock ${canonical.endpoint.name.lowercase()} hostname does not match the selected `${endpoint.name.lowercase()}` endpoint."
)
}
if (region != null && canonical.region != region) {
throw OpenAIException(
"The Bedrock endpoint region `${canonical.region}` does not match the configured AWS region `$region`."
)
}
}

private fun normalizeBaseUrl(value: String): String =
try {
value.toHttpUrl().toString().removeSuffix("/")
Expand Down Expand Up @@ -309,6 +396,7 @@ private class BearerAuthenticator(

private class SigV4Authenticator(
baseUrl: String,
private val endpoint: BedrockEndpoint,
private val region: Region,
private val credentialsProvider: AwsCredentialsProvider,
private val ownsCredentialsProvider: Boolean,
Expand Down Expand Up @@ -367,7 +455,7 @@ private class SigV4Authenticator(
payload(ContentStreamProvider.fromByteArray(bodyBytes))
}
}
.putProperty(AwsV4HttpSigner.SERVICE_SIGNING_NAME, BEDROCK_SERVICE)
.putProperty(AwsV4HttpSigner.SERVICE_SIGNING_NAME, endpoint.signingService)
.putProperty(AwsV4HttpSigner.REGION_NAME, region.id())
.putProperty(HttpSigner.SIGNING_CLOCK, clock)
}
Expand Down Expand Up @@ -422,14 +510,15 @@ private class SigV4Authenticator(
}

private fun validateCanonicalRegion(url: HttpUrl) {
val canonicalRegion =
Regex("^bedrock-mantle\\.([a-z0-9-]+)\\.api\\.aws$", RegexOption.IGNORE_CASE)
.matchEntire(url.host)
?.groupValues
?.get(1)
if (canonicalRegion != null && canonicalRegion != region.id()) {
val canonical = parseBedrockEndpointHostname(url.host) ?: return
if (canonical.endpoint != endpoint) {
throw OpenAIException(
"The Bedrock ${canonical.endpoint.name.lowercase()} hostname does not match the selected `${endpoint.name.lowercase()}` endpoint."
)
}
if (canonical.region != region.id()) {
throw OpenAIException(
"The Bedrock endpoint region `$canonicalRegion` does not match the SigV4 region `${region.id()}`."
"The Bedrock endpoint region `${canonical.region}` does not match the SigV4 region `${region.id()}`."
)
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package com.openai.bedrock

/** Selects an Amazon Bedrock endpoint family and its matching AWS SigV4 signing service. */
enum class BedrockEndpoint {
/** Uses the existing Bedrock Mantle endpoint and the `bedrock-mantle` signing service. */
MANTLE,

/** Uses the regional Bedrock Runtime endpoint and the `bedrock` signing service. */
RUNTIME;

internal val signingService: String
get() = if (this == RUNTIME) "bedrock" else "bedrock-mantle"
}
Loading
Loading