Models and Routing¶

Which models the gateway offers, where they come from, and what it does with a model name a client sends. Part of the Configuration Guide.

Settings Summary¶

Application Behavior¶

Model Sources and Access¶

Which model catalogues the gateway draws from, and which model identifiers a request is allowed to name.

AWS_BEDROCK_MANTLE_PREFERRED_MODELS¶

Purpose : Model IDs (or ID prefixes) served by Amazon Bedrock Mantle even when also available on the classic bedrock-runtime endpoint

Type : Comma-separated string of model IDs or ID prefixes

Default : openai.gpt-5.6 (the OpenAI GPT-5.6 family; every other dual-homed model is served by bedrock-runtime)

Behavior : Useful to leverage Mantle's independent throughput quotas, native response storage or built-in server tools for selected models. Mantle quotas (per-model, per-region tokens-per-minute) are independent from bedrock-runtime quotas.

: The GPT-5.6 family is preferred by default because Amazon Bedrock serves its web_search and code_interpreter tools on Mantle alone — on the classic endpoint they can only be refused.

: An explicit value replaces the default rather than adding to it: repeat openai.gpt-5.6 to keep the family on Mantle while preferring other models too.

export AWS_BEDROCK_MANTLE_PREFERRED_MODELS='openai.gpt-5.6,anthropic.claude-haiku-4-5'

export AWS_BEDROCK_MANTLE_PREFERRED_MODELS=

AWS_BEDROCK_EXTERNAL_WEB_ACCESS¶

Purpose : Let the built-in web search tool reach the public web

Type : Boolean

Default : false

Behavior : Controls whether the built-in web search tool may reach the external web. Searches are answered from the Amazon Bedrock web index and cache either way, and answers are current and carry source citations. AWS documents that retrieval is served entirely from that index and cache today, so no request data leaves the AWS boundary even when this is enabled, and that a future release may allow live external retrieval — at which point request data may leave it. Enabling it is therefore a decision taken in advance about behaviour that can change. It also requires the bedrock-websearch:ExternalWebAccess IAM permission on the credentials this server uses; each action is authorized only when a model actually attempts it, and a denied call does not fail the request.

export AWS_BEDROCK_EXTERNAL_WEB_ACCESS=true

AWS_BEDROCK_ALLOW_EXTERNAL_WEB_ACCESS_OVERRIDE¶

Purpose : Allow a request to override AWS_BEDROCK_EXTERNAL_WEB_ACCESS with the external_web_access extra model parameter

Type : Boolean

Default : false

Behavior : When true, a request that sends external_web_access as an extra model parameter decides for that request, on the models whose web search takes a web access choice per request — the OpenAI GPT-5.x family. When false, a request that sets it to anything other than the configured value is rejected with 400 rather than being silently overridden; a request that omits it always gets the configured value. A request asking for a value a model cannot be given is rejected with 400 as well, rather than accepted and quietly ignored. On the models that do take it, a request naming no web search tool is accepted: nothing is searched, so the value has no search to apply to.

export AWS_BEDROCK_ALLOW_EXTERNAL_WEB_ACCESS_OVERRIDE=true

AWS_BEDROCK_LEGACY¶

Purpose : Allow usage of legacy/deprecated Bedrock models

Type : Boolean

Default : false

export AWS_BEDROCK_LEGACY=true

AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK¶

Purpose : Transparently reroute requests using a deprecated model ID to its recommended replacement

Type : Boolean

Default : true

Behavior : When true, any request that specifies a deprecated model ID (as listed in the server's deprecation registry) is silently retried with the recommended replacement model. The replacement is fully re-evaluated — alias resolution, modality checks, and region routing all apply to the new model ID. When false, deprecated model IDs return a 404 error with a message indicating the replacement, forcing clients to migrate explicitly.

# Transparent fallback (default) — clients using old model IDs keep working
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=true

# Strict mode — deprecated model IDs return 404, clients must update their code
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=false

AWS_BEDROCK_DEPRECATED_MODELS¶

Purpose : Extend or override the built-in deprecated model registry with custom mappings

Type : JSON object — dict[str, str]

Default : {}

Behavior : Merged with the built-in registry at startup. User-provided entries take precedence over built-in ones — this means it can be used both to add new deprecated model mappings and to override the fallback target of an already-defined deprecated model. Effective only when AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK is true.

Reference : Amazon Bedrock model lifecycle

# Add a custom deprecated model and override an existing built-in mapping
export AWS_BEDROCK_DEPRECATED_MODELS='{"my-old-model-v1": "my-new-model-v2", "amazon.titan-text-lite-v1": "amazon.nova-lite-v1:0"}'

AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE¶

Purpose : Control automatic subscription to new models in AWS Marketplace

Type : Boolean

Default : true

Behavior : When true, the server automatically subscribes to new models discovered in the AWS Marketplace, making them immediately available through the API. When false, only models with existing marketplace subscriptions are visible and accessible

IAM Permissions Required : aws-marketplace:Subscribe, aws-marketplace:ViewSubscriptions — see Marketplace Auto-Subscribe IAM

# Allow automatic subscription (default)
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=true

# Restrict to pre-subscribed models only
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=false

AWS_BEDROCK_MARKETPLACE_ENDPOINTS_ENABLED¶

Purpose : Publish the Amazon Bedrock Marketplace model endpoints deployed in this account and serve them as ordinary chat models

Type : Boolean

Default : false

Behavior : When true, the server discovers the Marketplace model endpoints deployed in this account and publishes them in the model list. This is serve-only: the server never creates, updates or deletes an endpoint — it only invokes ones the operator already deployed. An endpoint Amazon Bedrock has not yet registered, or that SageMaker has not brought into service, is not published; it appears at the next model-cache refresh once it is

IAM Permissions Required : bedrock:ListMarketplaceModelEndpoints, bedrock:GetMarketplaceModelEndpoint, sagemaker:InvokeEndpoint, sagemaker:InvokeEndpointWithResponseStream — see Bedrock Marketplace Model Endpoints IAM

# Disabled (default) - Marketplace model endpoints are never published
# No environment variable needed

# Publish Marketplace model endpoints deployed in this account
export AWS_BEDROCK_MARKETPLACE_ENDPOINTS_ENABLED=true

AWS_BEDROCK_MARKETPLACE_ENDPOINT_REGIONS¶

Purpose : Restrict which regions are searched for Amazon Bedrock Marketplace model endpoints

Format : Comma-separated string of AWS region codes

Default : Every AWS_BEDROCK_REGIONS entry

Behavior : Every region listed here must also appear in AWS_BEDROCK_REGIONS, or the server refuses to start: a model endpoint is invoked in its own region and has no cross-region form, so a region the server does not otherwise serve could never answer

# Search only these regions for Marketplace model endpoints
export AWS_BEDROCK_MARKETPLACE_ENDPOINT_REGIONS=eu-west-1,us-east-1

AWS_BEDROCK_ALLOW_MARKETPLACE_ENDPOINT_ARN¶

Purpose : Allow users to pass the ARN of an Amazon Bedrock Marketplace model endpoint directly as a model ID in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use a Marketplace model endpoint ARN instead of a model ID in the model parameter, including an endpoint that is not published in the model list. When disabled, only the published model IDs are accepted

# Disabled (default) - users can only use published model IDs
# No environment variable needed

# Enable Marketplace model endpoint ARN support
export AWS_BEDROCK_ALLOW_MARKETPLACE_ENDPOINT_ARN=true

arn:aws:sagemaker:us-east-1:123456789012:endpoint/my-endpoint

AWS_SAGEMAKER_ENDPOINTS¶

Purpose : Publish Amazon SageMaker AI endpoints you run as chat models

Format : JSON object, mapping the model ID clients name to that endpoint's declaration

Default : Empty — no SageMaker AI endpoint is served

Behavior : Each entry publishes one endpoint in the model list and serves it on the chat completions, responses and messages APIs. The server only invokes the endpoints you name: it never creates, updates, scales or deletes one. Fields per entry:

| Field | Required | Meaning |
|---|---|---|
| `endpoint` | yes | Endpoint name — the name, never the ARN |
| `region` | yes | AWS Region the endpoint lives in |
| `inference_component` | for component-hosted endpoints | Inference component name; required for a scale-to-zero endpoint, which is always component-hosted |
| `name` | no | Display name in the model list (defaults to the model ID) |
| `provider` | no | Provider in the model list (defaults to `Amazon SageMaker AI`) |
| `input_modalities` | no | Input modalities to advertise, `["TEXT"]` by default; add `IMAGE` only when the model and its container accept image content parts |

IAM Permissions Required : sagemaker:CallWithBearerToken, sagemaker:InvokeEndpoint — see SageMaker AI Endpoints IAM

# Publish one endpoint as the model ID "my-qwen3"
export AWS_SAGEMAKER_ENDPOINTS='{
  "my-qwen3": {
    "endpoint": "my-endpoint",
    "region": "us-east-1",
    "inference_component": "my-inference-component",
    "name": "Qwen3 1.7B",
    "provider": "Qwen"
  }
}'

AWS_SAGEMAKER_WARMUP_TIMEOUT¶

Purpose : How long a request may wait for a SageMaker AI endpoint that has scaled to zero to provision capacity again

Type : Integer (seconds)

Default : 600

Behavior : An endpoint scaled to zero has no capacity to answer with, and the request that finds it cold is what makes AWS provision an instance again. Rather than surfacing that as an error, the server holds the connection and retries until the endpoint answers or this budget runs out; concurrent callers to the same endpoint share one wait. When the budget runs out the caller gets a 503 telling them to retry, and the real cause is logged for you. Set to 0 to disable the wait and answer that same 503 immediately — a retriable status, because a cold endpoint is a transient condition. It cannot exceed AI_RESPONSE_TIMEOUT, or the server refuses to start

# Wait up to 10 minutes for a cold endpoint (default)
# No environment variable needed

# Fail immediately instead of waiting
export AWS_SAGEMAKER_WARMUP_TIMEOUT=0

AWS_SAGEMAKER_ENDPOINT_URL¶

Purpose : Override the Amazon SageMaker AI runtime endpoint URL

Format : HTTPS URL template, with {region} substituted per Region

Default : Resolved automatically, per AWS partition

Behavior : Only needed to reach the runtime through a VPC endpoint or an inspection proxy. Must use https://, and the {region} placeholder must be well formed, or the server refuses to start

# Reach SageMaker AI through an interface VPC endpoint
export AWS_SAGEMAKER_ENDPOINT_URL='https://vpce-0123-abcd.runtime.sagemaker.{region}.vpce.amazonaws.com'

Model Cache¶

stdapi.ai automatically discovers and caches available Bedrock models from configured regions. Once the cache expires, the request that notices is answered from the cached list straight away and the refresh runs in the background — see Model List Refresh for what that means for freshness.

MODEL_CACHE_SECONDS¶

Purpose : Age at which the cached Bedrock model list is refreshed

Type : Integer (seconds, must be greater than 0)

Default : 900 (15 minutes)

Behavior : Once the cached list reaches this age, the next request needing it (a model lookup, /v1/models, /search_models) is answered from the cached list and a refresh starts in the background: the server queries Amazon Bedrock to discover newly available models, pick up model access changes, and update inference profile configurations. This cache also applies to application inference profile and prompt router information when users pass ARNs directly (if enabled via AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN or AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN)

# Default: 15 minutes
export MODEL_CACHE_SECONDS=900

# More frequent updates (5 minutes)
export MODEL_CACHE_SECONDS=300

# Less frequent updates (1 hour)
export MODEL_CACHE_SECONDS=3600

MODEL_CACHE_MAX_STALE_SECONDS¶

Purpose : Maximum age the cached model list may reach while its refresh keeps failing

Type : Integer (seconds, 0 or more)

Default : 86400 (24 hours)

Behavior : Below this age, an expired list is served while its refresh runs behind it. At or beyond it, the next request waits for a successful refresh instead — so a deployment whose refreshes fail silently (revoked bedrock:ListFoundationModels, a prolonged regional outage) cannot serve an arbitrarily old list, and a model that has been withdrawn stops being advertised. Each failed refresh is recorded in the server log, at error level once the list is more than two MODEL_CACHE_SECONDS old

# Default: 24 hours
export MODEL_CACHE_MAX_STALE_SECONDS=86400

# Tighter bound for a deployment where model availability changes matter
export MODEL_CACHE_MAX_STALE_SECONDS=3600

# Never serve an expired list: every expiry is refreshed synchronously
export MODEL_CACHE_MAX_STALE_SECONDS=0

MODEL_CACHE_SHARED¶

Purpose : Share one model list between the servers of a deployment

Type : Boolean

Default : false

Behavior : When enabled, the model list is published to the Amazon DynamoDB table named by AWS_DYNAMODB_TABLE, which is required. One server refreshes the list and publishes it; the others read it instead of querying Amazon Bedrock themselves, so a fleet performs one discovery pass per MODEL_CACHE_SECONDS instead of one per server, and a server that starts serves requests without a discovery pass of its own. Any table error is recorded in the server log and the server falls back to querying Amazon Bedrock itself

export AWS_DYNAMODB_TABLE=stdapi-ai
export MODEL_CACHE_SHARED=true

Default Model Parameters¶

Configure default inference parameters applied automatically to specific models.

DEFAULT_MODEL_PARAMS¶

Purpose : Per-model default parameters

Format : JSON object with model IDs as keys

Supported Parameters:

Only the outer JSON shape (an object of per-model objects) is validated at startup. The parameter values above are validated lazily, the first time a model with configured defaults is used: a wrong type, or a value below the lower bounds shown in the table, fails that request with HTTP 400. The numeric ceilings (for example the usual top_p maximum of 1.0) are enforced by Amazon Bedrock and the target model.

Configuration Examples¶

Basic Parameters:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-micro-v1:0": {
    "temperature": 0.3,
    "max_tokens": 800
  }
}'

Provider-Specific Features:

export DEFAULT_MODEL_PARAMS='{
  "anthropic.claude-sonnet-5": {
    "anthropic_beta": ["Interleaved-thinking-2025-05-14"]
  }
}'

Multiple Models:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-micro-v1:0": {
    "temperature": 0.3,
    "max_tokens": 500
  },
  "amazon.nova-lite-v1:0": {
    "temperature": 0.7,
    "max_tokens": 2000
  },
  "anthropic.claude-sonnet-5": {
    "temperature": 0.5,
    "top_p": 0.9,
    "anthropic_beta": ["Interleaved-thinking-2025-05-14"]
  }
}'

Advanced Configuration:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-pro-v1:0": {
    "temperature": 0.7,
    "top_p": 0.95,
    "max_tokens": 4096,
    "stop_sequences": ["Human:", "Assistant:"]
  }
}'

Parameter Merging¶

graph LR
    A[Default Parameters] --> B[Merged Config]
    C[Request Parameters] --> B
    B --> D[Final Configuration]

1. Default parameters are applied first (from DEFAULT_MODEL_PARAMS)
2. Request parameters override defaults if both are specified
3. Provider-specific fields are forwarded to Bedrock as additional model request fields
4. Unsupported fields reach Bedrock as-is, and a field the model rejects surfaces as a ValidationException returned to the client as HTTP 400. Three cases are handled before that: anthropic_beta flags are filtered individually against an allowlist (see ANTHROPIC_BETA_FILTER); a system prompt sent to a model that does not support one is dropped when DROP_UNSUPPORTED_SYSTEM_PROMPT is enabled (the default); and Amazon Nova 2 drops max_tokens when reasoning effort is high, logging a warning

Default Model Service Tiers¶

Configure default service tiers applied automatically to specific Bedrock models.

DEFAULT_MODEL_SERVICE_TIERS¶

Purpose : Per-model default service tier

Format : JSON object with model IDs as keys and tier string as value

Default : {}

Supported Values:

AWS_BEDROCK_ALLOW_SERVICE_TIER_OVERRIDE¶

Purpose : Control whether clients can select the service tier at request level

Default : true (clients may select a tier)

Cost Consideration : Service tiers are billed at different rates. Set to false on a shared deployment to pin every model to the tier you configured, so a client cannot move its traffic to a more expensive tier. A model with no configured tier still honors the request in either case.

Scope : Applies to models served through the Bedrock Converse and InvokeModel APIs. A Bedrock Mantle-served model carries no configured tier, so its requests always run on the tier they name.

export AWS_BEDROCK_ALLOW_SERVICE_TIER_OVERRIDE=false

Configuration Examples¶

Single Model:

export DEFAULT_MODEL_SERVICE_TIERS='{
  "amazon.nova-pro-v1:0": "flex"
}'

Multiple Models:

export DEFAULT_MODEL_SERVICE_TIERS='{
  "amazon.nova-pro-v1:0": "flex",
  "amazon.nova-premier-v1:0": "priority"
}'

Service Tier Merging¶

For models served through the Bedrock Converse and InvokeModel APIs:

1. Explicit request parameter takes highest priority
2. HTTP header (X-Amzn-Bedrock-Service-Tier, see Per-Request Service Tier Configuration) overrides defaults
3. Tier configured on the requested alias (see Model Aliases) applies if the request sets none
4. Default from DEFAULT_MODEL_SERVICE_TIERS applies if neither does
5. No service tier passed to Bedrock if unset

Steps 1 and 2 are skipped when AWS_BEDROCK_ALLOW_SERVICE_TIER_OVERRIDE is false and a tier is configured.

On a Bedrock Mantle-served model, only step 1 applies: the request's own service_tier is forwarded as sent, and neither the header, nor an alias' tier, nor DEFAULT_MODEL_SERVICE_TIERS takes part.

Model Aliases¶

Configure custom aliases to map user-friendly model names to actual model IDs. This enables OpenAI API compatibility and simplifies model references.

MODEL_ALIASES¶

Purpose : Map alias names to actual model IDs or ARNs

Format : JSON object with alias names as keys, and as values either a model ID or ARN, or an object carrying that model plus the configuration to apply to it

Default : {} (empty, uses built-in defaults only)

Aliases That Carry Configuration¶

An alias may map to an object instead of a model name. Every request naming that alias then gets the configuration attached to it, so one deployment can publish the same model under several names with different tiers, safeguards or defaults.

Every field below is designed around an Amazon Bedrock model call. An alias pointing at a model served by another AWS service — Amazon Polly, Amazon Transcribe, Amazon Comprehend — still resolves the name, and two fields keep working there: extra_params applies wherever the route accepts model parameters, and guardrail_id is enforced on the routes that check content through Amazon Bedrock Guardrails (speech input, transcripts) — a billed guardrail evaluation. service_tier and metadata configure the Bedrock call itself and are ignored on those services.

export MODEL_ALIASES='{
  "support-assistant": {
    "model": "amazon.nova-lite-v1:0",
    "service_tier": "flex",
    "guardrail_id": "abc123def456",
    "guardrail_version": "1",
    "metadata": {"team": "support"},
    "extra_params": {"temperature": 0.2}
  }
}'

Configuration Examples¶

Basic Alias:

export MODEL_ALIASES='{
  "my-tts": "amazon.polly-neural",
  "my-stt": "amazon.transcribe"
}'

Override Default Aliases:

# Override the default tts-1 mapping
export MODEL_ALIASES='{
  "tts-1": "amazon.polly-generative"
}'

Multiple Custom Aliases:

export MODEL_ALIASES='{
  "fast-model": "amazon.nova-micro-v1:0",
  "balanced-model": "amazon.nova-lite-v1:0",
  "quality-model": "amazon.nova-pro-v1:0",
  "claude": "anthropic.claude-sonnet-5"
}'

Map OpenAI Models to Bedrock:

# Make OpenAI model names work with Amazon Bedrock models
export MODEL_ALIASES='{
  "gpt-5": "anthropic.claude-sonnet-5",
  "gpt-4o": "anthropic.claude-sonnet-5",
  "gpt-4o-mini": "anthropic.claude-haiku-4-5-20251001-v1:0",
  "dall-e-3": "amazon.nova-canvas-v1:0",
  "dall-e-2": "stability.stable-image-ultra-v1:1"
}'

Override Deprecated Models:

# Redirect deprecated model IDs to their newer replacements
export MODEL_ALIASES='{
  "amazon.titan-image-generator-v1": "amazon.nova-canvas-v1:0",
  "amazon.titan-text-express-v1": "amazon.nova-lite-v1:0",
  "anthropic.claude-3-5-sonnet-20240620-v1:0": "anthropic.claude-sonnet-5",
  "stability.stable-image-ultra-v1:0": "stability.stable-image-ultra-v1:1"
}'

Advanced Routing with ARNs:

# Map friendly names to Application Inference Profiles or Prompt Routers
export MODEL_ALIASES='{
  "my-router": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/cost-optimizer",
  "my-profile": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123xyz"
}'

Using Aliases in API Requests¶

Once configured, aliases can be used anywhere a model ID is expected:

# Using the default tts-1 alias
curl https://api.example.com/v1/audio/speech \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tts-1",
    "input": "Hello world",
    "voice": "alloy"
  }'

# Using a custom alias
curl https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fast-model",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Alias Resolution¶

graph LR
    A[Model Name] --> B{Exact Model ID?}
    B -->|Yes| F[Resolved Model]
    B -->|No| C{Exact Alias?}
    C -->|Yes| F
    C -->|No| D{Deprecated Model?}
    D -->|Yes| F
    D -->|No| E{Wildcard Pattern?}
    E -->|Yes| F
    E -->|No| G[404 Not Found]

1. An exact model ID wins outright, a model served through Bedrock Mantle included.
2. An exact alias resolves next — a built-in default or one set in MODEL_ALIASES. An alias configured with a name that happens to look like a wildcard pattern — claude-*, say — is still a plain alias, and it wins here, before pattern resolution is ever tried; worth knowing, because it is easy to configure by accident.
3. A deprecation replacement applies to a name AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK covers.
4. A wildcard pattern resolves last, only once none of the above named a model.

The resolved model is validated and used for the request; a name matching none of the above answers 404. The response names the concrete model that served the request, never the alias or the pattern it was sent as — except POST /v1/moderations, which echoes the model value the caller sent, by design.

Model Wildcard Patterns¶

Anywhere a request names a model, it may name a glob pattern instead of an exact name, and the server serves the most recently released match — the same release date GET /search_models publishes as start_of_life_time and GET /v1/models publishes as created for each model.

• Glob syntax only — * and ?, case-sensitive; no regular expressions, no character classes ([...] is refused with 400, not treated as a class or as a literal). claude-sonnet-*, claude-opus-* and amazon.nova-* all work.
• At least three characters before the first * or ?. A broader pattern is refused with 400, and a bare * is never accepted.
• At most 255 characters. A longer pattern is refused with 400.
• Matched against both model IDs and aliases.
• Scoped to the endpoint called — the same pattern can resolve to a different model on chat than on embeddings.
• Resolved once, when the request is accepted. A batch job created with a pattern is pinned to the model that pattern meant that day, and reports that concrete model for its whole life.
• Never a legacy model, a model AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK covers, or a model whose first use would open a paid Marketplace subscription. Name one of those explicitly and it still resolves exactly as it does today; a pattern only ever skips over it.
• A model whose release date the server cannot order is never selected by a pattern, and its presence never makes an otherwise-unique match ambiguous — a property of that model, not of any particular catalogue. On POST /v1/audio/speech every model is one of these — Polly carries no release date — so a pattern there is accepted but never matches anything, and always answers 404.

Use GET /search_models?model=<pattern> to see everything a pattern matches, newest first, before relying on it in a request — the recommended way to check what a pattern will do.

Two routes take a concrete model only, never a pattern: POST /v1/moderations resolves the model before the request is examined, and POST /v1/realtime/client_secrets fixes the model into the ephemeral token before a connection exists — the realtime WebSocket endpoint's own model parameter does accept a pattern.

System Prompt Handling¶

Control how system prompts are handled for models that don't support them.

DROP_UNSUPPORTED_SYSTEM_PROMPT¶

Purpose : Control system prompt behavior for models that don't support system prompts

Type : Boolean

Default : true

# Default: silently drop system prompts for unsupported models
export DROP_UNSUPPORTED_SYSTEM_PROMPT=true

# Strict mode: return error when system prompt is used with unsupported model
export DROP_UNSUPPORTED_SYSTEM_PROMPT=false

Anthropic Beta Flag Filtering¶

Anthropic-compatible clients like Claude Code send anthropic-beta headers with experimental beta flags. Many of these flags (such as files-api-2025-04-14, prompt-caching-2024-07-31) are not supported by Amazon Bedrock and cause ValidationException errors (HTTP 400).

stdapi.ai automatically filters out unsupported flags while preserving supported ones, so clients work without any special configuration. Previously, the workaround was to set CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 on the client side, but this also disabled Bedrock-supported flags like Interleaved-thinking-2025-05-14 and token-efficient-tools-2025-02-19, degrading capabilities. This workaround is no longer needed.

Filtering is controlled by two settings: ANTHROPIC_BETA_FILTER to enable or disable it, and ANTHROPIC_BETA_ALLOWLIST to extend the built-in set of allowed flags.

ANTHROPIC_BETA_FILTER¶

Purpose : Enable or disable filtering of unsupported anthropic_beta flags for Anthropic Claude models

Type : Boolean

Default : true

Behavior : When enabled, anthropic_beta flags not in the allowlist are silently removed from requests before they reach Bedrock. A warning is logged when flags are filtered. When disabled, all flags are passed through to Bedrock as-is

# Enabled (default) - filter unsupported flags automatically
# No environment variable needed

# Disable filtering entirely (pass all flags through to Bedrock)
export ANTHROPIC_BETA_FILTER=false

ANTHROPIC_BETA_ALLOWLIST¶

Purpose : Add extra anthropic_beta flags to the built-in set of Bedrock-supported flags

Format : Comma-separated string of additional beta flag names

Default : Empty (only the built-in Bedrock defaults are used)

Behavior : The flags specified here are merged with the built-in set of Bedrock-supported flags. You only need to specify extra flags beyond the defaults (e.g., newly added Bedrock flags). Only effective when ANTHROPIC_BETA_FILTER is true

# Use built-in defaults only (recommended) - no environment variable needed

# Add newly supported Bedrock flags without waiting for a stdapi.ai update
export ANTHROPIC_BETA_ALLOWLIST='new-feature-2026-03-01,another-flag-2026-04-01'

Built-in Allowed Flags:

EXTRA_MODEL_PARAMS_DENYLIST¶

Purpose : Add extra parameter names to strip from the "extra model parameters" passthrough (any undeclared top-level JSON field on a chat or non-chat route, forwarded to Bedrock as a provider-specific inference field)

Format : Comma-separated string of additional parameter names

Default : Empty (only the built-in denylist is used)

Behavior : The names specified here are merged with the built-in denylist of LiteLLM client-control parameters (such as drop_params, api_key, custom_llm_provider) that some OpenAI-SDK-based clients leak into extra_body and that are never legitimate Bedrock model parameters — for example RAGFlow hardcodes extra_body={"drop_params": True} on every embeddings call, which previously reached Bedrock as an unrecognized inference field and failed with ValidationException. Every other extra parameter keeps being forwarded as before. Only effective when EXTRA_MODEL_PARAMS_DROP_ALL is false

# Use the built-in denylist only (recommended) - no environment variable needed

# Also strip a project-specific control field some client leaks into requests
export EXTRA_MODEL_PARAMS_DENYLIST='x_internal_debug_flag,x_proxy_trace_id'

EXTRA_MODEL_PARAMS_DROP_ALL¶

Purpose : Disable the "extra model parameters" passthrough entirely

Type : Boolean

Default : false

Behavior : When enabled, no undeclared request field is ever forwarded to Bedrock as a provider-specific inference parameter, on every route that supports the passthrough (chat completions/responses/messages, and embeddings/images/audio/rerank/etc.). This overrides EXTRA_MODEL_PARAMS_DENYLIST: with drop-all enabled, denylist filtering no longer matters because nothing is forwarded. Per-model defaults configured through DEFAULT_MODEL_PARAMS are unaffected — only request-supplied extras are dropped

# Keep the passthrough (default) - no environment variable needed

# Lock the deployment down to only declared API fields
export EXTRA_MODEL_PARAMS_DROP_ALL=true

Image Generation¶

IMAGE_GENERATION_MODEL¶

Purpose : Default Bedrock image model ID used when the image_generation integrated tool is invoked via the Responses API. The tool intercepts requests from any text model, generates the image against this Bedrock image model, and returns an image_generation_call output item.

Type : String (Bedrock image model ID)

Default : None — the tool returns HTTP 400 if no model is configured and the request does not specify one

Behavior : The tool definition in the request may include a model field to override this default per call. Priority: request model field > this env var. Any available Bedrock image generation model can be used — for example amazon.nova-canvas-v1:0, amazon.titan-image-generator-v2:0, or the Stability AI Stable Image / Stable Diffusion family. Legacy models (such as amazon.titan-image-generator-v1 and stability.stable-diffusion-xl-v1) are hidden unless AWS_BEDROCK_LEGACY is enabled. Use the Search Models API to list the image models available in your deployment.

export IMAGE_GENERATION_MODEL='amazon.nova-canvas-v1:0'

With this set, any text model can generate images via the Responses API:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Generate a sunset over the ocean.",
    "tools": [{"type": "image_generation"}],
    "tool_choice": "required"
  }'