Chat Completions API¶

Generate conversational AI responses with Amazon Bedrock foundation models—including Claude, Nova, Llama, and more—through an OpenAI-compatible interface.

Why Choose the Chat Completions API?¶

Quick Start: Available Endpoint¶

Feature Compatibility¶

Stored Chat Completions¶

Set store: true to persist a chat completion in Amazon Bedrock session storage — same mechanism, region, and KMS setting as stored responses. The returned id then works with the full stored-completion surface:

• GET /v1/chat/completions — list stored completions, sorted by creation time (order, after, limit), filterable by model and by metadata pairs. Metadata filters accept either one metadata[key]=value parameter per key (what the OpenAI SDKs send) or a single metadata={"key": "value"} JSON object of string values, for clients that can only send a whole object in one query parameter. A bare metadata in any other shape is rejected with 400 naming both accepted forms.
• GET /v1/chat/completions/{completion_id} — retrieve the stored completion.
• POST /v1/chat/completions/{completion_id} — replace its metadata (null clears it).
• GET /v1/chat/completions/{completion_id}/messages — list its input messages.
• DELETE /v1/chat/completions/{completion_id} — delete it and its backing session.

store defaults to false on this implementation and is ignored with stream=true or when the server lacks the session storage IAM permissions (a warning is recorded in the request log). Listings scan a capped number of sessions (1,000) in the primary Bedrock region; accounts beyond the cap may see incomplete listings.

Model Support¶

All models supported by the Amazon Bedrock Converse and Converse Stream API are supported, plus every model served by Bedrock Mantle when enabled — including OpenAI GPT-5.x, xAI Grok, and Google Gemma 4. Requests to Mantle models are passed through natively or converted automatically depending on the model's upstream API support — see Bedrock Mantle below.

Bedrock Mantle¶

Mantle-served requests follow one of three paths, each with its own parameter fidelity:

Model Name Aliases¶

This API supports dynamic model name aliases matching official provider APIs. Models like OpenAI and Anthropic provide dynamic aliases in their official APIs—this gateway supports the same model names, automatically resolving them to Amazon Bedrock model identifiers.

Examples (OpenAI GPT OSS models supported by Bedrock):

• gpt-oss-20b → openai.gpt-oss-20b-1:0

Advanced Features¶

Prompt Caching¶

Reduce costs and improve response times by caching frequently-used prompt components across multiple requests. This feature is particularly effective for applications with consistent system prompts, tool definitions, or conversation contexts.

Supported Models:

• Anthropic Claude: Full support for system, messages, and tools caching
• Amazon Nova: Support for system and messages caching

How to Use:

Set the prompt_cache_key parameter to enable caching:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-fable-5",
    "prompt_cache_key": "default",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant with extensive knowledge..."
      },
      {"role": "user", "content": "What is 2 + 2?"}
    ]
  }'

Granular Cache Control:

Enable caching for specific prompt sections using dot-separated values:

• "system" - Cache system messages only
• "messages" - Cache conversation history
• "tools" - Cache tool/function definitions (Anthropic Claude only)
• "system.messages" - Cache both system and messages
• "system.tools" - Cache system and tools
• "messages.tools" - Cache messages and tools
• "system.messages.tools" - Cache all components
• Any other non-empty value - Cache all components

{
  "model": "anthropic.claude-fable-5",
  "prompt_cache_key": "system.tools",
  "messages": [...],
  "tools": [...]
}

Benefits:

• Cost Reduction: Cached tokens are billed at a lower rate than regular input tokens
• Lower Latency: Cached prompts eliminate reprocessing time
• Automatic Management: The API handles cache invalidation and updates

Cache Retention (TTL):

Control how long cached prompts persist using the prompt_cache_retention parameter:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-fable-5",
    "prompt_cache_key": "default",
    "prompt_cache_retention": "24h",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant..."
      },
      {"role": "user", "content": "What is 2 + 2?"}
    ]
  }'

Available Retention Values:

• "in_memory" - Short-term caching (mapped to 5 minutes on Amazon Bedrock)
• "24h" - Long-term caching (mapped to 1 hour on Amazon Bedrock)
• Additional Amazon Bedrock values: "1h", "5m" (provider-specific)

The OpenAI prompt_cache_options object is also accepted: its ttl ("30m") is mapped to a 1 hour Amazon Bedrock retention on Anthropic models (other models use the default 5 minute TTL) when prompt_cache_retention is not set.

Explicit Cache Breakpoints:

Instead of relying on the prompt_cache_key section heuristics, mark the exact cache boundaries with prompt_cache_breakpoint on any content part (text, image_url, input_audio, file, refusal). Each marked part is followed by an Amazon Bedrock cachePoint, so the prompt prefix ending with that part is cached:

{
  "model": "anthropic.claude-fable-5",
  "prompt_cache_options": {"mode": "explicit"},
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "Long reusable instructions...",
          "prompt_cache_breakpoint": {"mode": "explicit"}
        }
      ]
    },
    {"role": "user", "content": "What is 2 + 2?"}
  ]
}

• "mode": "explicit" caches only the marked parts: the prompt_cache_key heuristics are disabled for that request.
• "mode": "implicit" (default) keeps the prompt_cache_key heuristics and honors the marked parts.
• At most 4 cache points are sent per request (Amazon Bedrock limit); the oldest ones are dropped when more are requested.
• Breakpoints on models without prompt caching support are accepted and ignored, as are breakpoints on tool result messages — those never become a cache point, whatever the model.

Usage Tracking:

Cached token usage is reported in the response:

{
  "usage": {
    "prompt_tokens": 1500,
    "completion_tokens": 100,
    "total_tokens": 1600,
    "prompt_tokens_details": {
      "cached_tokens": 1200,
      "cache_write_tokens": 300
    }
  }
}

In this example, 1,200 tokens were retrieved from cache and the remaining 300 tokens were processed and written to the cache. cache_write_tokens (an extra field beyond the OpenAI API) reports the tokens written to the cache when the model reports them, on both non-streaming responses and the trailing stream_options.include_usage chunk; it is omitted when no cache write occurred.

System Prompt¶

System prompts define the AI assistant's behavior, personality, and instructions (e.g., "You are a helpful assistant"). Most models support system prompts.

S3 Image Support¶

Access images directly from your S3 buckets without generating pre-signed URLs or downloading files locally.

Supported Formats:

• Images: JPEG, PNG, GIF, WebP

How to Use:

Simply reference your S3 images using the s3:// URI scheme in image_url fields:

{
  "model": "anthropic.claude-fable-5",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image"},
        {
          "type": "image_url",
          "image_url": {"url": "s3://my-bucket/images/photo.jpg"}
        }
      ]
    }
  ]
}

Benefits:

• No pre-signed URLs - Direct S3 access without generating temporary URLs
• Security - Images stay in your AWS account with IAM-controlled access
• Performance - Optimized data transfer within AWS infrastructure
• Large images - No size limitations of data URIs or base64 encoding

Files API References (file-id:)¶

The string-overloaded image_url.url, file.file_data, and input_audio.data fields also accept the project-local file-id: URI scheme to reference files previously uploaded via the Files API:

{
  "model": "anthropic.claude-fable-5",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "Describe this image"},
      {"type": "image_url", "image_url": {"url": "file-id:file-0190c51c7de7455d9b8c2efe27dfbf67"}}
    ]
  }]
}

Amazon Bedrock System Tools¶

Amazon Bedrock system tools are built-in capabilities that foundation models can use directly, without requiring you to implement backend integrations.

How to Use:

Add system tools to your tools array as normal. System tools don't require parameter definitions—just specify the tool name and the model will handle the rest.

Amazon Nova Tools¶

Web Grounding¶

Amazon Nova Web Grounding enables models to search the web for current information, helping answer questions requiring real-time data like news, weather, product availability, or recent events. The model automatically determines when to use web grounding based on the user's query.

Usage:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-2-lite-v1:0",
    "messages": [
      {
        "role": "user",
        "content": "What are the current AWS Regions and their locations?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "nova_grounding"
        }
      }
    ]
  }'

Response Format:

When using web grounding, the API response includes annotations with URL citations in non-streaming mode:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The AWS Regions include...",
      "annotations": [
        {
          "type": "url_citation",
          "url_citation": {
            "url": "https://aws.amazon.com/about-aws/global-infrastructure/",
            "title": "AWS Global Infrastructure"
          }
        }
      ]
    }
  }]
}

Limitations:

• No streaming citations: URL citation annotations are not emitted in streaming responses.

export AWS_BEDROCK_MODEL_REGION_RESTRICT='{"amazon.nova-": ["us-east-1"]}'

Anthropic Claude Server Tools¶

Anthropic Claude models support server-side tools (bash, text editor, memory) that are executed by the model provider. Declare them using the standard OpenAI function tool format: set type to "function" and function.name to the tool name.

Supported Tools by Model:

Usage:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "Run a Python script that prints hello world."}
    ],
    "tools": [
      {"type": "function", "function": {"name": "bash"}},
      {"type": "function", "function": {"name": "str_replace_based_edit_tool"}}
    ]
  }'

Tool Parameters:

Some Claude server tools accept additional configuration. Pass tool-specific parameters inside function.parameters:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "Edit the file hello.py to print hello world."}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "str_replace_based_edit_tool",
          "parameters": {"type": "object", "max_characters": 5000}
        }
      }
    ]
  }'

Provider-Specific Parameters¶

Unlock advanced model capabilities by passing provider-specific parameters directly in your requests. These parameters are forwarded to Amazon Bedrock and allow you to access features unique to each foundation model provider.

How It Works:

Add provider-specific fields at the top level of your request body alongside standard OpenAI parameters. The API automatically forwards these to the appropriate model provider via Amazon Bedrock.

Examples:

Top K Sampling:

{
  "model": "anthropic.claude-fable-5",
  "messages": [{"role": "user", "content": "Write a poem"}],
  "top_k": 50,
  "temperature": 0.7
}

Configuration Options:

Option 1: Per-Request

Add provider-specific parameters directly in your request body (as shown in examples above).

Option 2: Server-Wide Defaults

Configure default parameters for specific models via the DEFAULT_MODEL_PARAMS environment variable:

export DEFAULT_MODEL_PARAMS='{
  "anthropic.claude-sonnet-5": {
    "anthropic_beta": ["context-management-2025-06-27"]
  }
}'

Behavior:

• Compatible parameters: Forwarded to the model and applied
• Unsupported parameters: Return HTTP 400 with an error message
• Reserved names: model_id and additional_request_fields collide with the gateway's own request-building parameters and are rejected with a 400 invalid_request_error naming the key, instead of being forwarded

Anthropic Claude Features¶

Enable cutting-edge Claude capabilities through Anthropic beta feature flags.

Beta Feature Flags¶

Enable experimental Claude features like interleaved thinking by adding the anthropic_beta array to your request (extended thinking itself is controlled through the reasoning parameters, not a beta flag):

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "messages": [{"role":"user","content":"Summarize the news headline."}],
    "anthropic_beta": ["Interleaved-thinking-2025-05-14"]
  }'

Reasoning Control¶

This API supports several approaches to control Amazon Bedrock reasoning behavior. Reasoning enables foundation models to break down complex tasks into smaller steps ("chain of thought"), improving accuracy for multi-step analysis, math problems, and complex reasoning tasks. All approaches work with all Amazon Bedrock models that support reasoning capabilities.

OpenAI and DeepSeek API-Compatible Reasoning Parameters¶

Use the reasoning_effort parameter with predefined effort levels. This format is shared by the OpenAI and DeepSeek Chat Completions APIs and works with all Amazon Bedrock models supporting reasoning.

Available Levels:

• none - Disable reasoning
• minimal - Quick responses with minimal reasoning
• low - Light reasoning for straightforward tasks
• medium - Balanced reasoning for most use cases
• high - Deep reasoning for complex problems
• xhigh - Maximum reasoning for complex problems
• max - Its own (higher) effort tier on the adaptive Claude models served by the Converse API (Sonnet/Opus 4.6 and later, plus Fable), which forward it unchanged; collapsed onto the model's top reasoning tier on the fixed-scale models (Claude 3.7 - 4.5, Amazon Nova 2, DeepSeek, Kimi). On Amazon Bedrock Mantle, Claude models — Mythos among them — are reached over the Anthropic Messages API, and that conversion maps max to high; every other Mantle-served model receives the level as sent. Claude 4.6 also maps xhigh down to high

What You Get:

• reasoning_content field (DeepSeek API-compatible): models include their thinking process in the response. The field name is an operator setting (CHAT_COMPLETIONS_REASONING_FIELD): reasoning_content by default, reasoning for clients written against OpenRouter or vLLM, or none to keep responses strictly OpenAI-shaped.
• Streaming support: choices[].delta.reasoning_content chunks in real time

Example:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-fable-5",
    "reasoning_effort": "high",
    "messages": [{"role": "user", "content": "Solve this complex problem..."}]
  }'

Qwen API-Compatible Reasoning Parameters¶

Use explicit enable_thinking & thinking_budget parameters for fine-grained control over thinking mode. This is a Qwen API-compatible format that works with all Amazon Bedrock models supporting reasoning.

Parameters:

• enable_thinking (boolean): Enable or disable thinking mode
  • Default: Model-specific (usually false)
• thinking_budget (integer): Maximum thinking process length in tokens
  • Only effective when enable_thinking is true
  • Passed to the model as budget_tokens
  • Default: Model's maximum chain-of-thought length

Example:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "enable_thinking": true,
    "thinking_budget": 2000,
    "messages": [{"role": "user", "content": "Solve this complex problem..."}]
  }'

Moonshot API-Compatible Thinking Control¶

The thinking parameter provides a Moonshot API-compatible format for controlling thinking/reasoning on models that support it.

Parameters:

• thinking={"type": "enabled"} — Enable thinking mode
• thinking={"type": "disabled"} — Disable thinking mode

This format is accepted for all reasoning-capable models. Models that don't support this parameter will ignore it.

OpenRouter API-Compatible Reasoning Object¶

The reasoning object groups the same controls into a single field, and include_reasoning toggles whether the reasoning text is returned.

Parameters:

• reasoning.effort (string): Effort level, exactly as reasoning_effort
• reasoning.max_tokens (integer): Reasoning token budget, exactly as thinking_budget (implies enabled)
• reasoning.enabled (boolean): Enable reasoning, exactly as enable_thinking
• reasoning.exclude (boolean): Omit the reasoning text from the response
• include_reasoning (boolean): false is equivalent to reasoning: {"exclude": true}

Example:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "reasoning": {"effort": "high", "exclude": true},
    "messages": [{"role": "user", "content": "Solve this complex problem..."}]
  }'

Replaying Reasoning in a Multi-Turn Conversation¶

Appending the assistant message you just received to messages — the standard multi-turn idiom, and what the DeepSeek API asks for after a tool call — is always accepted, reasoning_content included.

Anthropic Claude models only continue from a thinking passage they can recognise as their own, which a replayed text field is not, so their reasoning is left out of that turn instead. The message content, tool calls and refusal are sent unchanged and the conversation continues normally; only the earlier chain of thought is no longer visible to the model. Every other model family receives the replayed reasoning as-is.

What this does not affect, measured on Claude Haiku 4.5:

• Reasoning on the new turn. The model still thinks, and still returns reasoning_content — the reasoning setting for the turn being generated is independent of the history. It re-derives rather than continuing the earlier chain.
• Tool-call continuations. A turn that carried a tool call is answered correctly with the earlier reasoning left out.
• Prompt caching. Cache hits are unaffected, including when the cache point sits inside the conversation immediately after the turn whose reasoning is left out — measured across three turns, each one reading the previous turn's cache in full and extending it. The omission is the same on every turn, so the cached prefix stays identical and keeps growing.

Available Request Headers¶

This endpoint supports standard Bedrock headers for enhanced control over your requests. All headers are optional and can be combined as needed.

Content Safety (Guardrails)¶

Performance Optimization¶

Example with all headers:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Amzn-Bedrock-GuardrailIdentifier: your-guardrail-id" \
  -H "X-Amzn-Bedrock-GuardrailVersion: 1" \
  -H "X-Amzn-Bedrock-Trace: enabled" \
  -H "X-Amzn-Bedrock-Service-Tier: priority" \
  -H "X-Amzn-Bedrock-PerformanceConfig-Latency: optimized" \
  -d '{
    "model": "anthropic.claude-fable-5",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Model-Specific Features¶

TwelveLabs Pegasus¶

twelvelabs.pegasus-1-2-v1:0 is a video-understanding model. Because Pegasus accepts exactly one video and one text prompt per call, this API adapts the conversation automatically:

• The latest video found anywhere in the conversation (any role, any position) is forwarded as the video input.
• The latest contiguous run of user text (back to the previous assistant or tool turn) is concatenated and forwarded as the text prompt.
• temperature and max_tokens are forwarded.
• response_format: json_schema is forwarded as Pegasus's structured output.

Silently ignored (no error): system prompts, tools, top_p, stop sequences, and prompt caching.

Upstream format limitation: The OpenAI Chat Completions API has no video_url content part type. To stay fully compatible with standard OpenAI clients, pass the video as an image_url content part — the server detects the video MIME type automatically and routes it to Pegasus correctly.

Video input formats: data:video/mp4;base64,…, https://…, s3://bucket/key, or file-id:…. Videos above 18.75 MB are automatically uploaded to S3.

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "twelvelabs.pegasus-1-2-v1:0",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "image_url",
            "image_url": {"url": "s3://my-bucket/video.mp4"}
          },
          {"type": "text", "text": "Describe what happens in this video."}
        ]
      }
    ]
  }'

Try It Now¶

Basic chat completion:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "messages": [{"role": "user", "content": "Say hello world"}]
  }'

Streaming response:

curl -N -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "stream": true,
    "messages": [{"role": "user", "content": "Write a haiku about the sea."}]
  }'

Multi-modal with image:

{
  "model": "amazon.nova-micro-v1:0",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image"},
        {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}}
      ]
    }
  ]
}

With reasoning:

curl -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "reasoning_effort": "low",
    "messages": [{"role": "user", "content": "Solve 12*13"}]
  }'

Response with reasoning:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "reasoning_content": "12 × 10 = 120, plus 12 × 3 = 36 → 156",
      "content": "156"
    }
  }]
}

Ready to build with AI? Check out the Models API to see all available foundation models!