Responses API¶

Generate model responses with Amazon Bedrock foundation models through an OpenAI Responses API-compatible interface. Supports text, images, tool calling, and streaming.

At a glance¶

• Tools, with the full round trip. Function tools in, function_call items out, function_call_output back in — plus file search over Amazon Bedrock Knowledge Bases, web search and image generation as server-run tools.
• Streaming across the whole lifecycle. response.created through response.completed, with text and tool-argument deltas, reasoning deltas, web-search brackets and citation annotations as separate events.
• Structured output through text.format: json_object on every model, json_schema with schema validation on the models that support it.
• Extended reasoning through reasoning.effort, with the encrypted reasoning returned when include asks for it.
• State on the server, in your AWS account. store: true, previous_response_id, conversations and compaction are backed by Amazon Bedrock, not by the gateway.
• Differs from OpenAI: store defaults to false here, and response IDs are stdapi.ai's own — they carry the serving region and are not interchangeable with another provider's.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Say hello world"
  }'

Endpoints¶

Feature compatibility¶

Models¶

All models supported by the Amazon Bedrock Converse and Converse Stream API are supported, plus every model served by Bedrock Mantle when enabled — including OpenAI GPT-5.x, xAI Grok, and Google Gemma 4. Requests to Mantle models are passed through natively or converted automatically depending on the model's upstream API support.

Working with the Responses API¶

System Prompt (instructions)¶

Use instructions to define the assistant's behavior — it is injected as a Bedrock system block.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "instructions": "You are a helpful assistant that answers in French.",
    "input": "Say hello."
  }'

Function Tool Calling¶

Define function tools and submit results in a round-trip conversation.

Step 1 — Request a tool call:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "What'\''s the weather in Paris?",
    "tool_choice": "required",
    "tools": [
      {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    ]
  }'

Step 2 — Submit the tool result:

Without previous_response_id, each request is stateless: replay the full history, including the function_call item from step 1's output alongside its matching function_call_output.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": [
      {
        "role": "user",
        "content": "What'\''s the weather in Paris?"
      },
      {
        "type": "function_call",
        "call_id": "<call_id from step 1>",
        "name": "get_weather",
        "arguments": "{\"city\": \"Paris\"}"
      },
      {
        "type": "function_call_output",
        "call_id": "<call_id from step 1>",
        "output": "{\"temperature\": \"18°C\", \"condition\": \"cloudy\"}"
      }
    ],
    "tools": [
      {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    ]
  }'

Streaming¶

Real-time token streaming with granular SSE lifecycle events.

curl -N -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Tell me a short story.",
    "stream": true
  }'

The stream emits events in order: response.created → response.in_progress → response.output_item.added → response.content_part.added → response.output_text.delta (repeated) → response.output_text.done → response.content_part.done → response.output_item.done → a terminal event.

The terminal event matches the outcome, exactly like the OpenAI API:

• response.completed — generation finished normally.
• response.incomplete — output was cut short (e.g. max_output_tokens reached or content filtered); response.completed is not emitted, and the SDK's get_final_response() raises accordingly.
• response.failed — generation failed; the embedded response carries an error object.

If an error occurs mid-stream, a spec error event (with code, message, param, and sequence_number) is emitted, followed by a terminal response.failed snapshot.

Structured JSON Output¶

Request machine-readable output using text.format.

JSON object:

json_object is not schema-validated: the gateway asks the model to reply with a JSON object on your behalf, but nothing constrains its decoding, so this is a best-effort guarantee rather than a hard one. Including the word "JSON" in the input, as below, still helps the model comply. Use json_schema when the response must conform to a specific shape.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Return the current date and day of week as JSON.",
    "text": {"format": {"type": "json_object"}}
  }'

JSON schema:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "What is 2 + 2? Reply with answer and confidence.",
    "text": {
      "format": {
        "type": "json_schema",
        "name": "MathResult",
        "schema": {
          "type": "object",
          "properties": {
            "answer": {"type": "number"},
            "confidence": {"type": "number"}
          },
          "required": ["answer", "confidence"]
        }
      }
    }
  }'

Extended Reasoning¶

Enable chain-of-thought reasoning on supported models (e.g. Amazon Nova 2, Anthropic Claude 3.7+) via reasoning.effort.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai.gpt-5.6-sol",
    "input": "Solve: if a train travels 120 km in 90 minutes, what is its speed?",
    "reasoning": {"effort": "low"},
    "max_output_tokens": 4096
  }'

The chain of thought is returned as a reasoning output item preceding the assistant message. Where the text goes depends on the request:

Either way, the events sit between the item's response.output_item.added and .done, before the message events. Clients that only display summary, such as Codex by default, need reasoning.summary set to see the reasoning. On models that do not summarize their reasoning, the summary is the reasoning text they return. Token accounting depends on the serving path:

• Converse-served models: usage.output_tokens_details.reasoning_tokens is always 0; reasoning tokens are billed inside output_tokens.
• Mantle-native models: without a summary request, the reasoning is an encrypted-content item bracketed only by response.output_item.added / .done. usage.output_tokens_details.reasoning_tokens is the count the model reports.
• Mantle models converted to another API, Anthropic Claude included: the reasoning comes back as described above, with the token accounting of Converse-served models.

Claude Opus 4.7 and later, Sonnet 5, Fable and Mythos reason without returning the text unless the request sets reasoning.summary: their summarized reasoning then comes back in summary. The reasoning tokens are billed the same with or without it. The Chat Completions API has no summary request, so these models return no reasoning text there.

Add "include": ["reasoning.encrypted_content"] to attach an encrypted_content envelope to each reasoning item. Echo the item back in the input of the next request to carry the model's reasoning state (including signatures and redacted content) across turns with no server-side storage — reasoning items from the official OpenAI API are accepted too, with their encrypted content safely ignored. Echoing a reasoning item back without its encrypted_content replays it unsigned: Anthropic Claude models only continue from a thinking passage they can recognise as their own, so that reasoning is left out of the turn — the request still succeeds, only the earlier chain of thought is no longer visible to the model. Every other model family receives the unsigned reasoning as-is.

Prompt Caching¶

Prompt caching reduces latency and costs by caching repetitive prompt components. Moonshot Kimi K3 and OpenAI GPT-6 cache a repeated prompt prefix automatically, with no parameter — see Chat Completions prompt caching for how they treat the settings below. On the other models, set the prompt_cache_key parameter to enable:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai.gpt-5.6-sol",
    "instructions": "You are a helpful assistant.",
    "input": "What is Python?",
    "prompt_cache_key": "default"
  }'

Granular Cache Control:

Use dot-separated values to cache specific components:

• "system" — Cache system messages only
• "messages" — Cache conversation history
• "tools" — Cache tool/function definitions (Anthropic Claude only)
• "system.messages" — Cache both system and messages
• "system.tools" — Cache system and tools
• "messages.tools" — Cache messages and tools
• "system.messages.tools" — Cache all components
• Any other non-empty value — Cache all components

Example — Cache system and tools:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "instructions": "You are a data analysis assistant.",
    "input": "Analyze this dataset: ...",
    "tools": [{"type": "function", "name": "run_sql", ...}],
    "prompt_cache_key": "system.tools"
  }'

Benefits:

• Cost Reduction: Cached tokens are billed at a lower rate than regular input tokens
• Lower Latency: Cached prompts eliminate reprocessing time
• Automatic Management: The API handles cache invalidation and updates

Cache Retention (TTL):

Control how long cached prompts persist using the prompt_cache_retention parameter:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai.gpt-5.6-sol",
    "input": "Hello",
    "prompt_cache_key": "default",
    "prompt_cache_retention": "24h"
  }'

Valid values: in_memory (default), 24h, 1h, or 5m. The 1h and 5m values are Amazon Bedrock-specific. On Amazon Bedrock, in_memory maps to 5 minutes and 24h maps to 1 hour. The prompt_cache_options.ttl value "30m" is mapped to a 1 hour retention on Anthropic models (other models use the default 5 minute TTL) when prompt_cache_retention is unset.

Explicit Cache Breakpoints:

Instead of relying on the prompt_cache_key section heuristics, mark the exact cache boundaries with prompt_cache_breakpoint on any input content part (input_text, input_image, input_file). Each marked part is followed by an Amazon Bedrock cachePoint, so the prompt prefix ending with that part is cached:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "prompt_cache_options": {"mode": "explicit"},
    "input": [
      {
        "role": "user",
        "content": [
          {
            "type": "input_text",
            "text": "Long reusable context...",
            "prompt_cache_breakpoint": {"mode": "explicit"}
          },
          {"type": "input_text", "text": "Summarize it."}
        ]
      }
    ]
  }'

• "mode": "explicit" caches only the marked parts: the prompt_cache_key heuristics are disabled for that request.
• "mode": "implicit" (default) keeps the prompt_cache_key heuristics and honors the marked parts.
• At most 4 cache points are sent per request (Amazon Bedrock limit); the oldest ones are dropped when more are requested.
• Breakpoints on models without prompt caching support are accepted and ignored, as are breakpoints on tool output items — those never become a cache point, whatever the model — and a cache point that would directly follow a reasoning block, which Amazon Bedrock refuses.
• prompt_cache_options is echoed back on the response object.

Cached token usage is reported in the response:

{
  "usage": {
    "input_tokens": 1500,
    "input_tokens_details": {
      "cached_tokens": 1200
    },
    "output_tokens": 300,
    "total_tokens": 1800
  }
}

Following OpenAI semantics, input_tokens covers the full prompt: tokens read from and written to the cache are included, and cached_tokens (the tokens read from cache) is a subset of input_tokens. In this example, 1,200 of the 1,500 input tokens were retrieved from cache.

Managed Prompt Templates¶

The prompt parameter references a prompt template stored in Amazon Bedrock Prompt Management. Amazon Bedrock renders the template server-side, so the request body carries only the variable values.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "prompt": {
      "id": "arn:aws:bedrock:us-east-1:123456789012:prompt/ABCDE12345",
      "version": "1",
      "variables": {"genre": "pop", "number": "3"}
    }
  }'

Requirements

• prompt.id must be an Amazon Bedrock prompt ARN. OpenAI-hosted prompt template IDs (pmpt_…) do not exist on this gateway and return 400.
• prompt.version is an Amazon Bedrock version number. It is appended to the ARN, and must not disagree with a version already present in prompt.id. Omit it to run the working draft.
• The prompt must be a TEXT prompt bound to a model that this server can serve, and model must be that exact model: it is the model used for response formatting and cost attribution. The error message names the model the prompt uses.
• prompt.variables values must be plain strings — Amazon Bedrock prompt variables only carry text, so image, file and structured content parts return 400.
• The prompt's region is derived from its ARN and must be a configured Amazon Bedrock region; cross-region failover is disabled for the request.
• The model must be served by the Converse API: Mantle native models return 400, as they have no Prompt Management equivalent.

Rejected Alongside prompt

The stored prompt version already provides the conversation, the system prompt, the tools and the inference parameters, so combining prompt with input, instructions, tools, tool_choice, text, temperature, top_p, max_output_tokens, reasoning or previous_response_id returns 400 instead of silently dropping them.

Streaming, store, moderation and guardrail headers remain available. The remaining request-level parameters (metadata, service_tier, prompt_cache_*, …) are accepted and echoed on the response, but not applied: the Bedrock call carries only the prompt resource and its variables.

File Search¶

file_search lets any chat model answer from the files you indexed in a vector store, with no change to your client code:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-2-lite-v1:0",
    "input": "How many vacation days do I get?",
    "tools": [{
      "type": "file_search",
      "vector_store_ids": ["vs_abc123"]
    }]
  }'

The model decides when to search and with which query. Each search it runs is reported as a file_search_call output item — carrying the queries used — ahead of the message it grounded. When streaming, that item is framed by response.file_search_call.in_progress, .searching and .completed, and the answer streams after it.

The retrieved passages are not returned unless you ask for them:

{
  "model": "amazon.nova-2-lite-v1:0",
  "input": "How many vacation days do I get?",
  "include": ["file_search_call.results"],
  "tools": [{"type": "file_search", "vector_store_ids": ["vs_abc123"]}]
}

Each result then carries its file_id, filename, text, score and the attributes stored with the file.

Narrowing the search:

OpenAI Integrated Tools¶

The Responses API supports OpenAI's built-in tool types, automatically mapped to the target model's native tools.

Nova Tools¶

Nova models support web search and code execution as integrated tools.

Web Search (web_search, web_search_preview):

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-2-lite-v1:0",
    "input": "What is the current version of Python?",
    "tools": [{"type": "web_search"}]
  }'

Code Interpreter (code_interpreter):

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-2-lite-v1:0",
    "input": "Calculate the first 10 Fibonacci numbers",
    "tools": [{"type": "code_interpreter"}]
  }'

OpenAI GPT Web Search¶

The OpenAI GPT-5.x and GPT-6 families answer web_search with the search tool built into Amazon Bedrock. The model decides when a question needs current information, runs one or more queries, and grounds its answer in what it finds.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai.gpt-5.6-luna",
    "input": "What are the most significant AWS launches announced this month?",
    "tools": [{"type": "web_search"}]
  }'

Each search appears as a web_search_call output item, and every grounded statement carries a url_citation annotation on the assistant message's output_text content, whose start_index and end_index delimit the answer text it supports. When streaming, the lifecycle arrives as response.web_search_call.in_progress / .searching / .completed, and each citation as a response.output_text.annotation.added event.

{
  "model": "openai.gpt-5.6-luna",
  "input": "What shipped this week?",
  "tools": [{"type": "web_search"}],
  "external_web_access": true
}

Image Generation¶

The image_generation integrated tool works with all text models — Claude, Nova, and any future model. The gateway intercepts the tool, lets the LLM compose the image prompt and parameters via a synthetic function call, then generates the image against a configured Bedrock image model and returns an image_generation_call output item to the client. Intermediate function_call items are suppressed.

Example — Generate an image:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Generate a photorealistic image of a red panda sitting on a tree branch.",
    "tools": [{"type": "image_generation"}],
    "tool_choice": "required"
  }'

The response contains an image_generation_call output item:

{
  "output": [
    {
      "type": "image_generation_call",
      "id": "img_abc123",
      "status": "completed",
      "result": "<base64-encoded PNG>"
    }
  ]
}

You can also specify image parameters in the tool definition:

{
  "type": "image_generation",
  "size": "1024x1024",
  "quality": "high",
  "output_format": "png"
}

Computer Use Not Supported¶

Provider-Specific Parameters¶

A top-level field this API does not declare is forwarded to the model as a provider-specific inference parameter, as on Chat Completions and Messages. A capability Amazon Bedrock exposes and the OpenAI API has no field for is therefore reachable without leaving this endpoint — the OpenAI SDK sends these through extra_body:

{
  "model": "anthropic.claude-sonnet-5",
  "input": "Write a poem about the sea",
  "top_k": 50
}

Server-wide defaults per model come from DEFAULT_MODEL_PARAMS, and a per-request value wins over them.

Behavior:

• Compatible parameters: forwarded to the model and applied
• Unsupported parameters: the backend refuses the request, returned as a 400
• Reserved names: additional_request_fields, max_tokens, model_id, stop_sequences, temperature, top_logprobs and top_p are the argument names the gateway binds when it builds the Bedrock call, so sending one as an extra is rejected with a 400 naming it instead of binding twice — use the declared max_output_tokens, temperature, top_p and top_logprobs fields
• Client-side control fields: names no provider treats as inference parameters (LiteLLM's drop_params among them) are dropped before the call. EXTRA_MODEL_PARAMS_DENYLIST extends that list, and EXTRA_MODEL_PARAMS_DROP_ALL disables the passthrough entirely

Input Token Counting¶

Count input tokens without generating a response. Useful for estimating costs or checking context-window fit before making a full response call.

Basic usage:

curl -X POST "$BASE/v1/responses/input_tokens" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Hello, how are you?"
  }'

Response:

{
  "object": "response.input_tokens",
  "input_tokens": 142
}

With instructions and tools:

curl -X POST "$BASE/v1/responses/input_tokens" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "What is the weather?",
    "instructions": "You are a helpful assistant.",
    "tools": [{"type": "function", "name": "get_weather", "description": "Get weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}]
  }'

conversation is accepted, and the conversation's items are counted ahead of input. What token counting does not take is in Limits and behaviour to know.

Exact and approximate counts. Every text model is counted, on this route and on /v1/messages/count_tokens, and search_models lists both routes for all of them. The count is exact for Claude models up to 4.6. Every other model gets an approximation that errs high: on everything measured — English prose, code, JSON, requests offering tools, eight scripts, emoji, digits and encoded data, across 27 tokenizers — it was never below the exact count.

• Claude 4.7 and later (Opus 4.7, Opus 5.5, Sonnet 5, Fable) are counted 7% to 70% above the exact figure, about 50% above typically. The count is closest when Claude Haiku 4.5 is enabled in your deployment.
• Other models are counted at least 5% above it, about 30% above typically — more for one-line prompts and for non-Latin scripts. A model whose tokenizer was never measured gets the highest weights of all.
• An image counts at an allowance that grows with its pixels, a PDF 3,000 tokens a page, and an image or file given only by URL, S3 URI or file ID an allowance from its size: it is never downloaded to be counted.
• On a model counted approximately, truncation: "auto" counts the whole input, an upper bound of what a response keeps.

The usage a response reports is always exact.

Stored Responses¶

Set store: true to persist a response in Amazon Bedrock session storage: one AWS-managed session per stored response, encrypted at rest (optionally with your own KMS key), with no state on the server itself.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "amazon.nova-micro-v1:0", "input": "Hello!", "store": true}'

The returned id then works with:

• GET /v1/responses/{response_id} — retrieve the stored response.
• GET /v1/responses/{response_id}/input_items — list the input items that produced it. Bedrock Mantle native storage does not serve input item listings: for Mantle-stored responses this returns 404 with an explanatory message.
• DELETE /v1/responses/{response_id} — delete it (and its Amazon Bedrock session).
• POST /v1/responses/{response_id}/cancel — for Mantle region-tagged IDs, proxied to Bedrock Mantle (background responses are cancellable upstream); for Bedrock-session-stored responses it fails with the OpenAI synchronous-response error since execution is synchronous.
• previous_response_id on a new request — continue the conversation: the stored input and output are automatically prepended to the new input (instructions are not carried over, per the OpenAI API). A tool loop therefore sends back only the function_call_output items: the function_call they answer, and any reasoning or built-in tool item from that turn, come from the stored response.

Conversation Compaction¶

Compact a long conversation into a single compaction item to keep multi-turn sessions within the context window. The model summarizes the provided input; the summary comes back as an opaque item that you include in the input of later requests instead of the full history.

curl -X POST "$BASE/v1/responses/compact" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-pro-v1:0",
    "input": [
      {"role": "user", "content": "..."},
      {"role": "assistant", "content": "..."}
    ]
  }'

Response (trimmed to the compaction item; output also echoes the conversation's message items before it):

{
  "id": "resp-...",
  "object": "response.compaction",
  "created_at": 1752000000,
  "output": [
    {"id": "cmp_...", "type": "compaction", "encrypted_content": "..."}
  ],
  "usage": {"input_tokens": 1500, "output_tokens": 220, "total_tokens": 1720}
}

Continue the conversation by sending the compaction item back, followed by new messages:

{
  "model": "amazon.nova-pro-v1:0",
  "input": [
    {"id": "cmp_...", "type": "compaction", "encrypted_content": "..."},
    {"role": "user", "content": "Next question..."}
  ]
}

Context Window Management¶

A long session eventually outgrows the model's context window. The OpenAI API's mechanisms are all served:

{
  "model": "anthropic.claude-haiku-4-5-20251001-v1:0",
  "input": [...],
  "context_management": [{"type": "compaction", "compact_threshold": 150000}]
}

Continue from a compacted response as with the OpenAI API: with previous_response_id, or by sending the output back followed by the next message. A compaction item a response produced stands for everything before it, so items sent ahead of it are not read again and the next turn starts from the compaction. Streamed, the item is announced with response.output_item.added at output_index 0 while the summary is produced, then completed with response.output_item.done, and the answer's items follow from index 1.

The output is capped when only it overflows. As with the OpenAI API, an input the window holds alone but not beside max_output_tokens is answered whatever truncation says, with the output limited to what the window leaves: the response still echoes the requested max_output_tokens, and an answer that runs out of room is incomplete with incomplete_details.reason: "max_output_tokens". On Writer Palmyra and Google Gemma such an input is trimmed under truncation: "auto", and otherwise refused with context_length_exceeded. A max_output_tokens filling the window on its own is refused with a plain 400 asking to lower it.

Model-Specific Features¶

TwelveLabs Pegasus¶

twelvelabs.pegasus-1-2-v1:0 is a video-understanding model. Because Pegasus accepts exactly one video and one text prompt per call, this API adapts the conversation automatically:

• The latest video found anywhere in the conversation (any role, any position) is forwarded as the video input.
• The latest contiguous run of user text (back to the previous assistant or tool turn) is concatenated and forwarded as the text prompt.
• temperature and max_output_tokens are forwarded.
• text.format: json_schema is forwarded as Pegasus's structured output.

Silently ignored (no error): system prompt, tools, top_p, stop sequences, and prompt caching.

Upstream format limitation: The OpenAI Responses API has no input_video content type in its stable spec. To stay fully compatible with standard OpenAI clients, pass the video as an input_image content item — the server detects the video MIME type automatically and routes it to Pegasus correctly.

Video input formats: data:video/mp4;base64,…, https://…, s3://bucket/key, or file-id:…. Videos above 18.75 MB are automatically uploaded to S3.

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "twelvelabs.pegasus-1-2-v1:0",
    "input": [
      {
        "type": "message",
        "role": "user",
        "content": [
          {"type": "input_image", "image_url": "s3://my-bucket/video.mp4"},
          {"type": "input_text", "text": "Describe what happens in this video."}
        ]
      }
    ]
  }'

Limits and behaviour to know¶

What is rejected with a 400. max_tool_calls, an input larger than the model's context window unless truncation: "auto" can trim it to fit (context_length_exceeded), a context_management entry whose type is not compaction or whose compact_threshold is not an integer of at least 1000, a compaction_trigger that is not the last input item, is sent twice or comes with max_output_tokens below 20000, conversation together with previous_response_id, stream=true on GET /v1/responses/{response_id}, the moderation parameter on a Mantle-served model, and a phase on an input message whose role is not assistant (unknown_parameter) — on an assistant message it is accepted and dropped rather than stored.

A compaction_trigger with nothing before it is refused. The OpenAI API answers it with compaction items; here it is a 400 reading There is no conversation to compact. Send the trigger after at least one user, assistant or tool item, or with a previous_response_id.

What is accepted and ignored on Converse-served models. background (execution is synchronous), stream_options, reasoning.context, reasoning.mode, text.verbosity, client_metadata, top_logprobs, include values other than the two honored ones, and include / starting_after on a stored-response retrieval. On Mantle-native models these are forwarded upstream instead — see Bedrock Mantle passthrough above.

Tools with no backend equivalent are dropped, not refused. computer, mcp, local_shell, custom, programmatic_tool_calling and the others leave the tool configuration before the request reaches the model, so the request succeeds and the model simply cannot call them; the full list, and what the model does instead, is under Function Tool Calling. file_search is the exception — it is served, from the vector stores the request names.

Token counting takes fewer parameters than a generation. previous_response_id is not supported on POST /v1/responses/input_tokens, since resolving it would change the count; truncation: "auto" is, and counts the input a response would keep once its oldest turns are dropped — without it, an input larger than the context window is counted, as upstream counts it, with an answer always above the window, as exact as any other count on that model (see Input Token Counting); personality (a token-counting-only schema field) and reasoning.context are accepted and ignored.

Request headers¶

This endpoint supports standard Bedrock headers for enhanced control over your requests. All headers are optional and can be combined as needed.

Content Safety (Guardrails)¶

Performance Optimization¶

Example with all headers:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Amzn-Bedrock-GuardrailIdentifier: your-guardrail-id" \
  -H "X-Amzn-Bedrock-GuardrailVersion: 1" \
  -H "X-Amzn-Bedrock-Trace: enabled" \
  -H "X-Amzn-Bedrock-Service-Tier: priority" \
  -H "X-Amzn-Bedrock-PerformanceConfig-Latency: optimized" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Hello!"
  }'

Try it¶

Streaming response:

curl -N -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-micro-v1:0",
    "input": "Write a haiku about the sea.",
    "stream": true
  }'

Multi-modal with image:

{
  "model": "amazon.nova-micro-v1:0",
  "input": [
    {
      "role": "user",
      "content": [
        {"type": "input_text", "text": "Describe this image"},
        {"type": "input_image", "image_url": "https://example.com/photo.jpg"}
      ]
    }
  ]
}

With reasoning:

curl -X POST "$BASE/v1/responses" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai.gpt-5.6-sol",
    "input": "Solve 12 × 13",
    "reasoning": {"effort": "low"},
    "max_output_tokens": 4096
  }'

Next steps¶

Next: Conversations API · Chat Completions API · Models API · Files API