Bedrock Features¶

The Amazon Bedrock capabilities the gateway turns on for you, and the ARNs it is allowed to invoke. Part of the Configuration Guide.

Settings Summary¶

Bedrock Advanced¶

Bedrock Guardrails¶

Amazon Bedrock Guardrails add content filtering and safety controls to model inputs and outputs. The configured guardrail also powers the OpenAI-compatible Moderations API (POST /v1/moderations); without one, that API falls back to inline guardrail checks in supported regions, then Amazon Comprehend.

Route Coverage¶

The configured guardrail applies to every route that serves a request directly. The Batch API is the exception: Amazon Bedrock batch inference cannot apply a guardrail, so a batch a configured guardrail would cover is refused rather than run unchecked. Chat routes use the native Bedrock integration; routes whose AWS backend has no guardrail mechanism enforce it through the ApplyGuardrail API: client-supplied text is checked as INPUT before the backend call and generated text as OUTPUT after it. On the Realtime API, where content streams both ways for as long as the session is open, the check runs per turn and an intervention ends the session — see Guardrail coverage for what that does and does not catch.

Global Configuration¶

AWS_BEDROCK_GUARDRAIL_IDENTIFIER¶

Purpose : ID of the Bedrock Guardrail to apply

Required : Yes (together with AWS_BEDROCK_GUARDRAIL_VERSION)

export AWS_BEDROCK_GUARDRAIL_IDENTIFIER=abc123def456

AWS_BEDROCK_GUARDRAIL_VERSION¶

Purpose : Version of the Bedrock Guardrail

Required : Yes (together with AWS_BEDROCK_GUARDRAIL_IDENTIFIER)

export AWS_BEDROCK_GUARDRAIL_VERSION=1

AWS_BEDROCK_GUARDRAIL_TRACE¶

Purpose : Trace level for guardrail evaluation

Options : disabled, enabled, enabled_full

Default : None (optional)

export AWS_BEDROCK_GUARDRAIL_TRACE=enabled

AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE¶

Purpose : Control whether users can override the global guardrail configuration at request level via HTTP headers

Default : false (disabled for security)

Security Consideration : When set to false (default) and a global guardrail is configured, only the global configuration is enforced, preventing users from bypassing or modifying safety controls. Set to true if you need to allow per-request guardrail customization to override the global configuration.

Auto-Enable Behavior : If no guardrail is configured at all — both AWS_BEDROCK_GUARDRAIL_IDENTIFIER and AWS_BEDROCK_GUARDRAIL_VERSION unset, and no model alias carrying one — this setting is automatically set to true at startup, allowing per-request guardrails when no policy is enforced.

export AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE=true

export AWS_BEDROCK_GUARDRAIL_IDENTIFIER=abc123def456
export AWS_BEDROCK_GUARDRAIL_VERSION=1
export AWS_BEDROCK_GUARDRAIL_TRACE=enabled
export AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE=false  # Default: prevent overrides

Per-Request Guardrail Configuration¶

Use HTTP headers to specify guardrail settings per request:

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-GuardrailIdentifier: abc123def456" \
  -H "X-Amzn-Bedrock-GuardrailVersion: 1" \
  -H "X-Amzn-Bedrock-Trace: enabled" \
  -d '{"model": "anthropic.claude-sonnet-5", "messages": [...]}'

Request Body Configuration¶

The amazon-bedrock-guardrailConfig object in the request body is supported for OpenAI Chat Completions compatibility.

Bedrock Session Storage¶

Requests with store=true on the Responses and Chat Completions APIs persist generations in Amazon Bedrock sessions. No environment variable is needed to enable this — it requires the Bedrock Session Storage IAM permissions.

AWS_BEDROCK_SESSION_ENCRYPTION_KEY_ARN¶

Purpose : KMS key ARN encrypting the Amazon Bedrock sessions that back stored responses and stored chat completions (store=true)

Default : None — sessions are encrypted with the AWS-managed key

Validation : Checked at startup: must be a KMS key ARN (arn:<partition>:kms:<region>:<account-id>:key/<key-id>).

export AWS_BEDROCK_SESSION_ENCRYPTION_KEY_ARN=arn:aws:kms:us-east-1:123456789012:key/abcd-...

AWS_BEDROCK_BATCH_ROLE_ARN¶

Purpose : AWS IAM service role that Amazon Bedrock batch inference assumes to read a batch's requests and write its results — required to enable the Batch API and the Message Batches API

Type : String — an IAM role ARN

Default : None — the batch endpoints answer 503 (529 on the Anthropic-compatible routes) and the server reports the disabled feature in its startup log

Behavior : The server passes this role when it submits a batch; Amazon Bedrock then reads the requests and writes the results with it. The role must be able to read and write every bucket configured with AWS_S3_BUCKET and AWS_S3_REGIONAL_BUCKETS, under AWS_S3_BATCHES_PREFIX.

Validation : Checked at startup: must be an IAM role ARN (arn:<partition>:iam::<account-id>:role/<name>). Leaving it unset is reported as a startup warning, never a failure.

export AWS_BEDROCK_BATCH_ROLE_ARN=arn:aws:iam::123456789012:role/stdapi-ai-batch

AWS_BEDROCK_USER_ROLE_ARN¶

Purpose : Run each end user's model calls under an AWS IAM role session of their own, so AWS reports Amazon Bedrock model usage per end user in Cost Explorer and in the Cost and Usage Report

Type : String — an IAM role ARN

Default : None — every request runs under the server's own identity, and AWS reports all model usage under it

Behavior : The server opens one short-lived session of this role per end user, caches it, and signs that user's model invocations with it. The identity is the authenticated caller when authentication is enabled, otherwise the identifier the request declares (safety_identifier or user on the OpenAI-compatible APIs, metadata.user_id on the Anthropic Messages API). Only model invocations are covered — guardrail evaluations, video generation, speech, transcription and translation keep the server's identity. A session that cannot be opened fails the request rather than falling back to the server's identity.

Validation : Checked at startup: must be an IAM role ARN (arn:<partition>:iam::<account-id>:role/<name>). The server also tries to assume it at startup and reports a warning — not a failure — when it cannot.

export AWS_BEDROCK_USER_ROLE_ARN=arn:aws:iam::123456789012:role/stdapi-ai-end-user

AWS_BEDROCK_USER_ROLE_SESSION_DURATION¶

Purpose : Lifetime of a per-end-user role session, in seconds

Type : Integer — 900 to 3600

Default : 3600

Behavior : Sessions are cached per end user and reopened shortly before they expire, so a longer lifetime means fewer AWS STS calls. The upper bound is imposed by AWS: the server itself runs under an assumed role, and a role session obtained from another role session cannot last longer than one hour, whatever the role's maximum session duration.

export AWS_BEDROCK_USER_ROLE_SESSION_DURATION=1800

AWS_BEDROCK_USER_ROLE_TAG_KEY¶

Purpose : Session tag key carrying the end user identity on each per-end-user role session

Type : String, or null to send no session tag

Default : user

Behavior : Activate this key as a cost allocation tag — in the AWS Billing console, under Cost allocation tags filtered by type IAM principal — to group Bedrock costs by end user in Cost Explorer. The same tag is testable in IAM policies as aws:PrincipalTag/<key>, so the role can be restricted per user. With no tag, end users are still distinguished by their role session name in the Cost and Usage Report.

Validation : Checked at startup: 1 to 128 characters over letters, digits, spaces and _ . : / = + - @; keys beginning with aws: are reserved by AWS and rejected.

export AWS_BEDROCK_USER_ROLE_TAG_KEY=end-user

AWS_BEDROCK_USER_ROLE_REQUIRE_IDENTITY¶

Purpose : Reject a model request that identifies no end user, instead of running it under the server's own identity

Type : Boolean

Default : false — such requests run under the server's identity, and their usage is reported under it

Behavior : Enable it so that no model usage escapes per-user attribution: a request carrying neither an authenticated caller nor an end user identifier is answered 400. Clients that never send one stop working, so enable it only once every client identifies its user — and note that some APIs, audio transcription among them, have no end user field at all, so on those it takes an authenticated caller. A real-time speech-to-speech session keeps for its whole life the identity it opened with, so while this is enabled it is refused rather than attributed to the server. Requires AWS_BEDROCK_USER_ROLE_ARN.

export AWS_BEDROCK_USER_ROLE_REQUIRE_IDENTITY=true

Bedrock Service Tier and Performance Configuration¶

Amazon Bedrock service tiers and performance configurations allow you to optimize AI workload performance and cost trade-offs. Configure latency optimization and throughput priority for your inference requests.

Service Tiers¶

Service tiers help you match AI workload performance with cost by selecting the appropriate throughput and latency characteristics:

• priority - Highest priority processing with guaranteed capacity and fastest response times. Best for latency-sensitive applications.
• default - Standard processing with balanced performance and cost. Suitable for most production workloads.
• flex - Cost-optimized processing with flexible scheduling. Best for batch jobs and non-time-sensitive workloads.
• reserved - Dedicated reserved capacity with a throughput commitment (requires an AWS contract). Best for predictable workloads needing an uptime guarantee.

Performance Configuration¶

Performance configuration allows you to optimize for latency:

• standard - Standard latency profile with balanced performance
• optimized - Optimized for lowest possible latency

Per-Request Service Tier Configuration¶

Configure service tier and performance settings per request using HTTP headers. These headers are available on all Bedrock-based routes (Chat Completions, Embeddings, Images). Server-side per-model defaults can be set with DEFAULT_MODEL_SERVICE_TIERS.

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: priority" \
  -H "X-Amzn-Bedrock-PerformanceConfig-Latency: optimized" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

curl -X POST https://api.example.com/v1/embeddings \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: flex" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-2-multimodal-embeddings-v1:0",
    "input": ["text 1", "text 2", "text 3"]
  }'

curl -X POST https://api.example.com/v1/images/generations \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: default" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-canvas-v1:0",
    "prompt": "A serene mountain landscape"
  }'

Audio and Text-to-Speech¶

DEFAULT_TTS_MODEL¶

Purpose : Default text-to-speech model when not specified in requests

Default : amazon.polly-standard

export DEFAULT_TTS_MODEL=amazon.polly-neural

DEFAULT_TTS_LANGUAGE¶

Purpose : Default language code for text-to-speech synthesis when using OpenAI voice names

Default : None (automatic language detection via Amazon Comprehend)

Behavior : When specified, this language is used instead of automatic detection. When not set, Amazon Comprehend detects the language automatically from the input text.

Valid Language Codes: Any Amazon Polly language code (e.g., en-US, fr-FR, es-ES, de-DE, ja-JP)

# Use English (US) for all TTS requests
export DEFAULT_TTS_LANGUAGE=en-US

# Use French for all TTS requests
export DEFAULT_TTS_LANGUAGE=fr-FR

Realtime API¶

REALTIME_CLIENT_SECRET_KEY¶

Purpose : Secret the Realtime API's ephemeral client secrets (POST /v1/realtime/client_secrets) are signed with

Type : String (any value)

Default : None — a signing key is derived from the configured API key instead

Behavior : Ephemeral client secrets are stateless: nothing is stored server-side, so any instance behind a load balancer verifies a secret minted by any other, as long as they all sign with the same key. By default that key is derived from the deployment's own API key, which is already shared across every instance — so this setting has nothing to add on a deployment that already configures one.

Set it explicitly on a deployment that runs with **no API key configured at all**: without either one, each instance falls back to a random key generated **per process**, and a client secret minted by one instance then fails to verify on any other — the symptom is an ephemeral secret rejected intermittently on a multi-instance deployment. Any value works, as long as every instance shares it.

export REALTIME_CLIENT_SECRET_KEY=a-value-shared-by-every-instance

REALTIME_ALLOW_SESSION_OVERRIDE¶

Purpose : Whether a client connecting with an ephemeral client secret may override the session configuration that secret carries

Type : Boolean

Default : true — the upstream behavior: the carried configuration is a default the client may change

Behavior : A minted secret carries a session configuration, and by default a client opening a session with it may name another model on the ?model= query string and replace any of that configuration with its own session.update — exactly as it can against the upstream API.

Set to `false` on a multi-tenant deployment, where the secret is the only thing constraining an untrusted browser or mobile client. The `model`, the `instructions` and `max_output_tokens` the secret was minted with are then final: connecting with a `?model=` naming a different model is refused before the session opens, and a `session.update` changing any of the three answers an `error` event. Everything else — voice, audio formats, turn detection, transcription — stays under the client's control.

export REALTIME_ALLOW_SESSION_OVERRIDE=false

REALTIME_WEBRTC_ENABLED¶

Purpose : Serve WebRTC calls on POST /v1/realtime/calls, terminating the media path — ICE, DTLS-SRTP, Opus — in the server process

Type : Boolean

Default : false — the endpoint answers 404 and only the WebSocket transport is served

Behavior : Enabling it requires the webrtc optional dependencies (stdapi[webrtc], shipped in the container images; glibc only — aiortc publishes no musl wheels) and the server refuses to start without them. It also requires a network path the default deployment does not have: WebRTC media is UDP on ephemeral ports, negotiated directly to the instance that answered the SDP offer, which no HTTP(S) load balancer can carry. See the deployment section for what the Terraform module's media mode provisions, and the transport documentation for the call-control and duration limits that come with it.

export REALTIME_WEBRTC_ENABLED=true

REALTIME_WEBRTC_STUN_SERVER¶

Purpose : STUN server the gateway queries to discover its public address, which is then advertised to WebRTC callers as an ICE candidate

Type : String — a STUN URI, e.g. stun:stun.l.google.com:19302

Default : None — only the host's own addresses are advertised

Behavior : Required whenever the server sits behind 1:1 NAT, which is exactly the shape of an ECS task with a public IP: the task sees only its private address, so without STUN the SDP answer advertises addresses no caller can reach — the exchange succeeds and the call carries no audio. Any public STUN server works; the queried server learns nothing but the deployment's public address.

export REALTIME_WEBRTC_STUN_SERVER="stun:stun.l.google.com:19302"

REALTIME_WEBRTC_TURN_SERVER¶

Purpose : TURN relay advertised to WebRTC callers, carrying the media of callers whose networks block UDP to arbitrary ports

Type : String — a TURN URI, e.g. turn:turn.example.com:3478?transport=udp, with REALTIME_WEBRTC_TURN_USERNAME and REALTIME_WEBRTC_TURN_PASSWORD carrying its long-term credentials (all three together, or none)

Default : None — callers on UDP-blocking networks cannot establish media

Behavior : The relay is yours to run — coturn is the usual choice — because AWS offers no managed TURN service. It needs its own public address and its own always-on capacity, which is part of why a media framework in front remains the recommended shape for demanding audiences.

export REALTIME_WEBRTC_TURN_SERVER="turn:turn.example.com:3478?transport=udp"
export REALTIME_WEBRTC_TURN_USERNAME="stdapi"
export REALTIME_WEBRTC_TURN_PASSWORD="..."

REALTIME_WEBRTC_ALLOW_PRIVATE_CANDIDATES¶

Purpose : Accept the ICE candidates a WebRTC caller offers on addresses that are not globally routable — private (RFC 1918), shared (RFC 6598), loopback and link-local ones

Type : Boolean

Default : false — those candidates are dropped from the offer, and an offer left with no candidate at all is refused with invalid_offer

Behavior : An SDP offer names the addresses the server sends its ICE connectivity checks to, and the caller posting it holds nothing more than an ephemeral client secret. Screening the offer is what keeps an untrusted caller from aiming those UDP probes at addresses inside the deployment's own VPC. Enable this only where callers legitimately share that network — a same-VPC service, an on-premises deployment, a LAN — and the exchange otherwise succeeds with no media path. Hostname and mDNS (.local) candidates are dropped either way, whatever this is set to: resolving one is itself a lookup on the deployment's network, and a browser that offers only mDNS candidates cannot establish media with this gateway regardless.

export REALTIME_WEBRTC_ALLOW_PRIVATE_CANDIDATES=true

ARN Access Controls¶

Which Amazon Bedrock ARNs a request may name as its model, and how an ARN is mapped onto a model identifier.

AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass cross-region inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use cross-region inference profile ARNs instead of model IDs in the model parameter. Cross-region inference profiles enable routing to multiple regions for better availability

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable cross-region inference profile ARN support
export AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-sonnet-5

AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass application inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use application inference profile ARNs instead of model IDs in the model parameter. Application inference profiles are custom routing configurations for specific use cases

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable application inference profile ARN support
export AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123xyz

AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN¶

Purpose : Allow users to pass prompt router ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use prompt router ARNs instead of model IDs in the model parameter. Prompt routers enable dynamic model selection based on prompt characteristics

IAM Permissions Required : bedrock:GetPromptRouter (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable prompt router ARN support
export AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN=true

arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router

AWS_BEDROCK_ALLOW_PROMPT_ARN¶

Purpose : Allow users to reference an Amazon Bedrock Prompt Management prompt ARN in the OpenAI Responses API prompt parameter

Type : Boolean

Default : false

Behavior : When enabled, prompt.id accepts a prompt ARN (with an optional prompt.version) and prompt.variables fill in the template. Amazon Bedrock renders the stored prompt, and the model it is bound to serves the request. When disabled, any prompt parameter is rejected with a 400 error

IAM Permissions Required : bedrock:GetPrompt (resolve the prompt's model) and bedrock:RenderPrompt (invoke it)

# Disabled (default) - the Responses API `prompt` parameter returns 400
# No environment variable needed

# Enable Prompt Management prompt ARN support
export AWS_BEDROCK_ALLOW_PROMPT_ARN=true

arn:aws:bedrock:us-east-1:123456789012:prompt/ABCDE12345:1

AWS_BEDROCK_MODEL_ARN_MAPPING¶

Purpose : Map standard model IDs to custom inference profile or prompt router ARNs for server-controlled routing

Format : JSON object with model IDs as keys and ARNs as values

Default : {} (empty, no mappings)

Behavior : When configured, the mapped ARN is used instead of the default cross-region inference profile when clients request the model by its standard ID. This provides centralized control over model routing without requiring client changes

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-custom-profile",
  "anthropic.claude-haiku-4-5-20251001-v1:0": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/cost-optimizer"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/production-profile"
}'

# Production: Use cost-optimized prompt router
export AWS_BEDROCK_MODEL_ARN_MAPPING='{"anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/prod-router"}'

# Development: Use standard cross-region profile
export AWS_BEDROCK_MODEL_ARN_MAPPING='{}'

Using Inference Profile and Prompt Router ARNs¶

stdapi.ai supports passing ARNs directly as model IDs in API requests, enabling advanced routing capabilities beyond standard model selection.

Overview¶

Instead of using standard model IDs like anthropic.claude-sonnet-5, you can pass ARNs that reference:

• Cross-Region Inference Profiles - AWS-managed multi-region routing
• Application Inference Profiles - Your custom routing configurations
• Prompt Routers - Intelligent dynamic model selection

Enabling ARN Support¶

By default, users can only pass standard model IDs. To allow ARN usage, enable the appropriate settings:

# Allow cross-region inference profile ARNs
export AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN=true

# Allow application inference profile ARNs
export AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN=true

# Allow prompt router ARNs
export AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN=true

Using ARNs in API Requests¶

Once enabled, users can pass ARNs directly in the model parameter:

Cross-Region Inference Profile Example:

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

Application Inference Profile Example:

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-custom-profile",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

Prompt Router Example:

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

Use Case Comparison¶

Best Practices¶

Required IAM Permissions¶

When using ARN-based routing, ensure your IAM role/user has the appropriate permissions:

{
  "Sid": "BedrockARNRouting",
  "Effect": "Allow",
  "Action": [
    "bedrock:GetInferenceProfile",
    "bedrock:GetPromptRouter"
  ],
  "Resource": "*"
}

See the IAM Permissions page for complete policy examples.