Releases & Roadmap¶

stdapi.ai is under active development with regular feature releases.

Recent Releases¶

See Release History below for the full changelog of all releases.

Latest: v1.16.1 – A maintenance update to v1.16.0, which added four new API surfaces — conversations, batches, vector stores and realtime speech — plus retrieval a model calls for itself, a vector store served from your own Amazon Bedrock knowledge base, long and streamed text-to-speech, live transcription, per-caller authentication, and end-user cost attribution on the AWS bill. See the full release notes below.

Roadmap (Tracked on GitHub)¶

Pending features and current deployment state are tracked on the GitHub Project.

Release History¶

v1.16.0 – Conversations, Batches, Vector Stores, Realtime Speech & Per-User Identity (with v1.16.1 maintenance update)¶

This release adds four API surfaces and finishes the speech story. New APIs: Conversations keep a thread server-side, so a client continues it by id instead of resending the history; the OpenAI Batch API and Anthropic Message Batches API run large request sets asynchronously at the discounted batch price; Vector Stores index and search files by meaning — or address a knowledge base you already run — with a model reaching either kind for itself through file_search; and the Realtime API holds a spoken conversation over one WebSocket. Speech: 100,000-character synthesis spoken as it is produced, live transcription needing no bucket, and Amazon Nova Sonic as the lowest-cost speech-to-text backend here. Identity per caller: Amazon Cognito tokens alongside or instead of the API key, published discovery so an agent authenticates itself, and per-user cost attribution reporting each end user's spend from the AWS invoice rather than an estimate.

New Required IAM Permissions

v1.16.0 adds one action every deployment needs, a handful that belong to statements you may already grant, and one statement per optional feature. See IAM Permissions for the policies in full.

Enough of them together that they no longer fit one policy: IAM caps a customer managed policy at 6,144 characters, and a deployment enabling most of these exceeds it. Attach several policies to the role rather than widening actions to save room — the Terraform module now ships two, one for Amazon Bedrock and one for the supporting services, and does that for you.

Required on upgrade, whatever the deployment does:

bedrock:InvokeModelWithBidirectionalStream — serves every model invoked over a two-way connection — the Realtime API, and Amazon Nova Sonic transcription and translation. It belongs to the core Bedrock policy; no route-specific action exists for any of them.

Add to a statement you already grant, if the deployment uses that feature:

bedrock:UpdateSession, on the session storage statement — the conversation metadata update (POST /v1/conversations/{id}) and nothing else. The rest of the Conversations API uses the actions stored responses already require.
bedrock-mantle:CountTokens, on the Bedrock Mantle statement — counts the tokens of a Mantle-served model on /anthropic/v1/messages/count_tokens, since Amazon Bedrock's own CountTokens takes Anthropic models only. Needed by any deployment serving Mantle models, which is the default.
transcribe:StartStreamTranscription, on the speech-to-text statement — serves stream=true on /v1/audio/transcriptions. Add it with the upgrade: without it a streamed request that names its language answers 503 feature_unavailable, and the server log names the permission. It stages nothing, so a deployment with no bucket at all grants this one alone.
polly:StartSpeechSynthesisStream, polly:StartSpeechSynthesisTask and polly:GetSpeechSynthesisTask, plus s3:PutObject, s3:GetObject and s3:DeleteObject on each bucket serving an Amazon Polly Region, on the text-to-speech statement — they serve input above 3,000 characters and nothing else. With a bucket configured and these missing, long requests are accepted and then fail on the permission: grant the whole set, or leave the bucket unconfigured and keep the 3,000-character answer.
translate:ListLanguages, on the translation statement — read once at startup so an unsupported language pair is refused before the audio is transcribed. Genuinely optional: without it the check stays off and translation still works, reporting the unsupported pair once the translation call itself fails.

New statements, one per optional feature:

Vector stores — s3vectors:CreateIndex, DeleteIndex, PutVectors, GetVectors, QueryVectors and DeleteVectors, scoped to your vector bucket and its indexes. No bucket-level create or delete is granted: the gateway creates and deletes the indexes inside the bucket, never the bucket.
Knowledge base vector stores — bedrock:GetKnowledgeBase, Retrieve, ListDataSources, IngestKnowledgeBaseDocuments, ListKnowledgeBaseDocuments, GetKnowledgeBaseDocuments and DeleteKnowledgeBaseDocuments, one statement per allowlisted knowledge base ARN. bedrock:ListKnowledgeBases is deliberately not granted and not needed — the server only ever addresses the identifiers it was given.
Batch inference — bedrock:CreateModelInvocationJob, GetModelInvocationJob and StopModelInvocationJob on the server's role, plus iam:PassRole conditioned on bedrock.amazonaws.com. The service role Amazon Bedrock assumes carries its own policy: s3:GetObject, s3:PutObject and s3:ListBucket on the batch prefix, and bedrock:InvokeModel on the models you batch.
Per-user cost attribution — sts:AssumeRole and sts:TagSession on the server's role and in the end user role's trust policy (both actions: without TagSession, every tagged session is denied), and bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream and bedrock:ApplyGuardrail on the end user role itself, since AWS authorizes those against the caller of the invocation.
Web search — bedrock-websearch:InvokeSearch and bedrock-websearch:InvokeFetch, plus bedrock-websearch:ExternalWebAccess only where a request may reach the open internet. Leaving that last one out is what keeps every search inside the AWS boundary. A missing web-search permission produces no error and no server log entry: the model answers without having searched, so check these before suspecting the model.
Transcription output encryption — kms:GenerateDataKey and kms:Decrypt on the key named by AWS_TRANSCRIBE_OUTPUT_ENCRYPTION_KEY_ARN, in the key policy as well as on the role.
Durable vector store indexing — sqs:SendMessage, sqs:ReceiveMessage, sqs:DeleteMessage, sqs:ChangeMessageVisibility and sqs:GetQueueAttributes, on the single queue named by AWS_SQS_VECTOR_STORE_QUEUE_URL and never on *. Needed only if you configure that queue; leave it unset and nothing here applies. No queue is ever created, deleted or reconfigured, so none of those actions is granted.

Five features stay inert until you create the resource they need

Nothing in this release is breaking — but these five answer 503, or stay off, until the resource exists in your own account:

Batches — an IAM service role Amazon Bedrock assumes to read the requests and write the results (AWS_BEDROCK_BATCH_ROLE_ARN), plus the bucket it reads and writes.
Vector stores — an Amazon S3 vector bucket you create yourself, and the Region it lives in.
Knowledge base vector stores — an allowlist of the knowledge bases this deployment may address (AWS_BEDROCK_KNOWLEDGE_BASE_IDS), empty by default. One that is not on it answers exactly as a store that does not exist, so the setting cannot be probed for what a deployment holds.
Cognito authentication — a user pool and its app clients (AWS_COGNITO_USER_POOL_ID); until then the API key remains the only method, exactly as before.
Per-user cost attribution — a role for the end user sessions (AWS_BEDROCK_USER_ROLE_ARN); off by default, and every call keeps being billed to the deployment's own identity until it is set.

Conversations, the Realtime API and streamed transcription need no new resource. Long speech input needs a bucket for the serving region, which is the same one the rest of the gateway already uses — except on generative voices, which speak up to 20,000 characters without one.

Behavior Changes

Review these before upgrading — they may change what existing clients or dashboards observe:

A missing deployment permission is no longer reported as the caller's. An AccessDeniedException on the gateway's own AWS calls reached clients as 403 permission_error — which every OpenAI and Anthropic SDK reads as their key being refused. Every route now answers 503 feature_unavailable, with the server log naming the operation, model and permission. Clients matching 403 for a backend permission error should match 503/feature_unavailable instead; a 403 now means only that per-user attribution is on and that end user's role was denied.
Built-in web search now appears in usage and cost reporting. Queries were recorded as nothing at all, so a measured turn under-reported its cost by 58%. Nothing AWS charges changed; what the gateway reports does. Web access is also an operator setting now (AWS_BEDROCK_EXTERNAL_WEB_ACCESS), defaulting to the previous behaviour.
A request that would be answered without what it asked for is refused. A /v1/responses web_search restricting its sources (filters.allowed_domains, user_location) was accepted and dropped, so answers came back sourced from domains the caller had excluded. Now a 400 on models that cannot serve it; Bedrock Mantle models receive the options unchanged. The same rule governs file search filters and score thresholds.
Two output-shaping hints that returned 400 now succeed. prediction and verbosity on chat completions are accepted and dropped, as the Responses surface already did; truncation="disabled" is likewise accepted, while truncation="auto" is still refused.
/v1/responses forwards undeclared request fields to the model, as chat completions and messages already did, so the backend may refuse one it does not recognise. Conversely, client-side control fields no provider treats as parameters (LiteLLM's drop_params among them) are dropped rather than forwarded. Both are governed by EXTRA_MODEL_PARAMS_DENYLIST and EXTRA_MODEL_PARAMS_DROP_ALL.
Attachments are measured against what the model actually accepts. The old guard compared raw bytes where the backend enforces base64 length, so it was ~33% too permissive. Oversized attachments are now staged and referenced where the model reads from storage, or refused with 413 naming the size it accepts. Smaller attachments are unaffected — see Attachment Size.
The server's own connections follow the proxy environment. HTTPS_PROXY, HTTP_PROXY and NO_PROXY were honoured by the AWS SDK and ignored by everything else, so a proxied deployment saw no Bedrock Mantle models. Two connections deliberately still bypass it: container metadata, and the fetch of a caller-supplied URL, where a proxy would defeat address validation. See proxied deployments.
A declared upload checksum is now verified. The value was stored and never looked at, so a corrupted upload completed like a clean one. It covers the file's contents, not the storage layer's multipart identifier — declaring the latter is now refused.
An unknown model name answers with a sentence, not the catalogue. The 404 body carried every served identifier, roughly 2,500 characters. Clients that parsed it for a model list should call /v1/models.
Bedrock Mantle is only probed in the Regions that serve it, so a deployment listing others no longer warns at every start. An explicit AWS_BEDROCK_MANTLE_REGIONS list is still used exactly as given.
The container health probe's command changed. Deployments that re-declare the probe instead of running the image's own — an ECS task definition among them — should take the command from the image.

New APIs¶

Provider	Endpoint/Feature	AWS Backend
OpenAI	`/v1/conversations` – create, retrieve, update and delete a conversation, list and manage its items, and continue it from the Responses API with the `conversation` parameter	Amazon Bedrock - session management
OpenAI	`/v1/vector_stores` – attach files, follow the indexing as it progresses, then search by meaning with attribute filters and per-passage scores	Amazon S3 Vectors, Amazon Bedrock - embedding models
OpenAI	`/v1/vector_stores` – address an Amazon Bedrock knowledge base you already run as a vector store, Bedrock managed or customer-managed: search it, attach, list, read and delete documents. Allowlisted per knowledge base, never created or deleted here	Amazon Bedrock - Knowledge Bases
OpenAI	`file_search` on `/v1/responses` – a chat model answers from the stores you name, reporting the searches it ran and citing a `file_citation` per file it drew on	Amazon S3 Vectors, Amazon Bedrock - Knowledge Bases
OpenAI	`/v1/batches` – run a JSONL file of chat completion or embedding requests asynchronously at the batch price: submit, poll, cancel, read the result files	Amazon Bedrock - batch inference
Anthropic	`/anthropic/v1/messages/batches` – the same asynchronous, batch-priced run for the Messages API, results streamed back as JSONL; each request may name its own model, up to eight per batch	Amazon Bedrock - batch inference
OpenAI	`WS /v1/realtime` – a live speech-to-speech session over one WebSocket, with a transcript of both sides, server-side turn detection or manual turns, barge-in, and G.711 for telephony	Amazon Bedrock - Amazon Nova Sonic
OpenAI	`POST /v1/realtime/client_secrets` – mint a short-lived, browser-safe credential carrying a session configuration; signed and stateless, so any instance verifies one minted by any other	Amazon Bedrock - Amazon Nova Sonic

Limits worth knowing before building on these

Realtime: a session lasts at most 8 minutes and calls no tools, and a spoken answer is guardrail-checked once complete, so a blocked one may already have been partly heard (coverage). WebRTC and SIP are not served — put LiveKit Agents or Pipecat in front for a browser media path or a phone line. The compatibility table lists every event the session does not emit.

Knowledge-base stores address a knowledge base that already exists and refuse, naming why, anything that would reshape it — creating, deleting, renaming, expiry, chunking strategy, attribute rewrites and the file-batch routes. Attaching needs a custom data source. Retrieval scores are reported as the backend states them rather than rescaled into similarities, and unknown values are reported unknown rather than invented. See Knowledge Base Stores.

Speech & Audio¶

Provider	Endpoint/Feature	AWS Backend
OpenAI	`/v1/audio/speech` – up to 100,000 billed characters per request, 24× the upstream 4,096, with no API change and no new request field	Amazon Polly
OpenAI	`/v1/audio/speech` – long input is spoken as it is synthesized instead of after a whole job finishes; generative voices reach 20,000 characters with no bucket at all, and each request takes whichever path can serve it, so long input is no longer tied to one voice	Amazon Polly
OpenAI	`/v1/audio/transcriptions` – naming Amazon Nova Sonic transcribes at the lowest cost available here, streamed as it is recognized; `json` and `text` only, up to 10 minutes, no timestamps. No existing request is re-routed	Amazon Bedrock - Amazon Nova Sonic
OpenAI	`/v1/audio/translations` – Amazon Nova Sonic translates speech to English itself, in one request	Amazon Bedrock - Amazon Nova Sonic
OpenAI	`/v1/audio/transcriptions` – `stream=true` returns each phrase as it is recognized, whenever the request names the language to expect; needs no bucket. `gpt-live-transcribe` is now an alias, and requests naming no language are unchanged unless `AWS_TRANSCRIBE_STREAM_LANGUAGES` says which to expect	Amazon Transcribe
OpenAI	`/v1/audio/transcriptions` – per-language custom vocabularies and language models, so a request identifying between several languages can apply the right resources to each one instead of being refused. Accepted only where the backend would use them: alongside a single fixed language, where they would apply to nothing, they are still refused	Amazon Transcribe
stdapi.ai	`AWS_TRANSCRIBE_OUTPUT_ENCRYPTION_KEY_ARN` – encrypt a transcription's output with a key you name rather than the bucket's own. The job's request identifiers travel as the encryption context, so a key policy can be scoped to this workload instead of to the whole bucket	Amazon Transcribe, AWS KMS
OpenAI	`/v1/audio/translations` – the supported language pairs are read once at startup and checked before the call, so a pair that cannot be served is named as the request problem it is instead of surfacing as a failure after the audio was transcribed. The permission that reads them is optional: without it the check stays off and everything else works	AWS Translate

Identity & Cost Attribution¶

Provider	Endpoint/Feature	AWS Backend
stdapi.ai	Amazon Cognito user pool tokens – accept access tokens instead of, or alongside, the API key, so each caller reaches the API with their own credential; validated in-process against the pool's published keys, with no AWS call on the request path	Amazon Cognito
stdapi.ai	`AUTHENTICATION_MODE` – assert the posture rather than infer it: the server refuses to start when the selected method is not configured, or when a configured method would be silently ignored	Amazon Cognito
stdapi.ai	Authentication discovery for agents – an OAuth 2.0 protected resource metadata document, pointed at by every unauthorized response, so an MCP client finds the authorization server and the scope it needs without being configured for this deployment; published only once an authorization server is declared	Amazon Cognito, or any OAuth 2.0 authorization server
stdapi.ai	Per-user cost attribution – model calls issued under a short-lived role session tagged with the caller, so AWS reports each end user's spend in Cost Explorer and the Cost and Usage Report, from the invoice rather than an estimate. Off by default; a deployment can also require every call to name its end user rather than bill it to the deployment	Amazon Bedrock, AWS STS
stdapi.ai	Vector store cost reporting – a search against a Bedrock-managed knowledge base is recorded and priced like every other billed unit; what cannot be accounted for is stated rather than approximated	Amazon Bedrock - Knowledge Bases

Platform Features¶

Feature	Description
Aliases that carry configuration	A `MODEL_ALIASES` entry may map a public name to the target model plus the service tier, guardrail, metadata and extra parameters applied to requests naming it, so one model is published under several names with different policies. The plain-string form is unchanged, and a malformed alias stops startup naming itself rather than failing once per request
Attachment size policy	On the multimodal routes served by Amazon Bedrock, an attachment is measured before the request is built and travels inline or by reference according to the limits each model class declares. Staging is per model and per media kind: of the families measured, only the Amazon Nova families and TwelveLabs Pegasus accept a reference
Ephemeral secret signing key	`REALTIME_CLIENT_SECRET_KEY` signs the Realtime API's client secrets. A deployment with an API key already shares one and needs nothing; one running with no API key at all should set it, or a secret minted by one instance fails to verify on another
Dual-stack container listener	The image's bind address moved out of its command into `GRANIAN_HOST`, so a deployment that needs a dual-stack socket — an ECS service whose discovery record includes an AAAA record, for instance — sets one variable instead of replacing the whole command. The IPv4-only default is unchanged
Faster container health probe	The probe ships as a module of the application itself, byte-compiled with the rest of the package and covered by the linters and the test suite; it speaks HTTP over a socket rather than pulling in 123 modules per probe, cutting roughly 250 ms of import work per run in the community image and halving its peak memory
OpenAI Daybreak models	Daybreak Red (GPT-5.6 Cyber) and Daybreak Blue (GPT-5.6 Sol) are served and priced with the rest of the GPT-5.6 family, image input included. Both answer on the Responses API through Bedrock's next-generation inference endpoint, in US East (Ohio) only, and both are gated on enrollment with OpenAI's Daybreak programme — an account without it does not see them in the catalogue at all
Capability discovery	The model catalogue advertises what this release added — speech to speech, its transcription and translation, the search surfaces, and whether a model can be used with the Batch API — filterable over HTTP and through the same tool an agent reads before it calls anything. Web search is credited to every model that provides it, not only to the family the last release added it for
Durable vector store indexing	`AWS_SQS_VECTOR_STORE_QUEUE_URL` hands indexing to an Amazon SQS queue you create, so a file keeps being indexed — and finishes — when the server that accepted it is replaced. Off by default; needs a standard queue with a dead-letter queue and the durable indexing permissions (resilience)
End-to-end client coverage	The suite driving complete, unmodified third-party clients against a live gateway gains five: LiteLLM, Docling Serve's vision pipeline, the OpenAI Agents SDK (realtime voice, conversations, web search, vector-store retrieval), and LiveKit Agents and Pipecat running the exact WebRTC and telephony configurations this documentation prints. Deployment guides follow for LobeHub and RAGFlow

Fixes¶

Vector store durability: an attached file whose indexing was interrupted is now reported failed with a last_error instead of sitting in_progress for ever; a detached file leaves every listing and search immediately and is gone only once its passages are, so a server replaced mid-delete no longer leaves a deleted document answering searches; indexing is bounded for the whole server rather than per request, so memory and embedding quota no longer scale with the number of callers; and deployments that would rather the work finished than reported can hand indexing to a queue
Work a request left running is finished before the server stops: a deployment, scale-in or Spot interruption used to drop temporary file cleanups, vector store indexing and live audio session releases with nothing in the logs. Shutdown now waits under SHUTDOWN_DRAIN_TIMEOUT (10 seconds by default), settles whatever the deadline leaves, and counts it in the stop event. Raise it together with your container runtime's kill delay, never one alone
Streamed responses run the work they scheduled: the drain was attached before the body produced a byte, so three leaks followed — a vector store searched only through streamed answers could expire mid-query, an expired store left a paid index behind, and a streamed transcription falling back to a job left its audio, transcript and job record behind on every request
Realtime speaks the released vocabulary, not the beta one: item events were unparsable, the caller's transcript event was dropped for a missing field, and the item lifecycle clients wait on was never emitted. Barge-in did not work at all — the session refused truncation precisely while an answer was playing, the only moment it is ever sent. Truncate, retrieve and delete now work against the tracked conversation, a written turn is answered instead of timing out, and every answer reports the six response fields upstream always sends
Batch API: listing a batch neither settled nor published it, so a client that only ever listed never had its usage recorded; every validation failure at submission was reported as an unsupported model, quota and role failures included; and the batch record was written only after the jobs started, leaving billable work running with nothing on disk to stop it
Knowledge-base stores answer in this API's own words: attaching to a managed knowledge base always failed, listing its files always failed, deleted documents never left a listing, and internal bookkeeping reached the caller as attributes. A file refused because the corpus is maintained elsewhere is now told so in the store's terms, and a refused file is explained by the store that refused it rather than by one fixed sentence describing limits the caller never met
Cost reporting matches what AWS charges: GPT-5.6 Luna was reported at five times its real cost and Terra a quarter over, since AWS repriced them on the model's own page rather than in the live price catalogue; a Global cross-Region call — the only way Luna, Sol and Terra are ever served — was costed at the In-Region rate, about 10% over; and moderation usage naming an alias matched no price at all. Every source page is now re-checked weekly
Reasoning and web search reach the models that serve them: Amazon Nova 2 and DeepSeek V3 refused the token budget the Anthropic dialect requires, leaving extended thinking unreachable on that route while the identical ask worked elsewhere; and a web_search sent to a GPT-5.6 model resolved to its non-Mantle twin travelled as an ordinary function tool, so no search ran and nothing said so — now a 400 naming both ways to route the model to the endpoint that serves it
The Messages surface reports what an answer cost and why it stopped: refusals carry the policy category, the reasoning-token breakdown is reported, and service tier and per-TTL cache-creation counts are populated — most visibly on a batch, which claimed no tier while being billed as one
Audio, embeddings and attachments are bounded correctly: inline audio was measured against raw bytes where the backend enforces the encoded length, so files between ~18.75 MB and 25 MB passed and were refused downstream; long text-to-speech now answers with the length a bucket-less deployment can honour; and models that embed one input per call no longer open a connection per chunk
The interactive documentation pages render with no outbound access: /docs and /redoc pulled the icon, Swagger UI, ReDoc and a web font from three third parties — blank pages in an air-gapped VPC, and elsewhere a report to those hosts of who was reading this API and when, running whatever a floating major version tag resolved to that day. Both are now served whole from the image, pinned to exact releases verified by SHA-256 at build time, with upstream licences beside them
MCP tools return what they produce: every route answering with bytes was published as a tool an agent could call and then could not use. An image now arrives as an image and audio as audio, anything the protocol cannot carry arrives as a reference rather than failing, and the 4 MiB body cap that blocked image edits now follows MAX_INPUT_FILE_SIZE
Addresses and listings are the ones this deployment serves: a custom route prefix still quoted default paths to search_models and to video job polling, neither recoverable client-side; and a file's created_at and its place in a listing came from two different clocks, so a multipart upload sat among older files reporting a later time. The Anthropic listing also answered oldest first, hiding every recent file, and now runs newest first as upstream does
Diagnostics name their cause: an unreachable Region rendered six different conditions as one identical sentence, and the slow startup beside it was the container metadata lookup retrying, unreported; a request abandoned mid-flight left its OpenTelemetry trace current, so later work was recorded under a closed request's trace id; and behind a proxy, client_ip recorded the load balancer whatever PROXY_TRUSTED_HOSTS allowed
The API describes itself, not the service behind it: a synthesis limit credited to the service enforcing it, prices credited to their catalogue and a moderation route naming the engine underneath all shipped in the OpenAPI document and in the tool descriptions agents read before calling

Fixes & Maintenance (v1.16.1)¶

Update cryptography to 50.0.1, rebuilt against OpenSSL 4.0.2

v1.15.0 – Reliability, Performance & Feature Completeness¶

This release focuses on making the whole gateway better rather than just bigger. Reliability and quality: the largest correctness pass to date — three successive deep audits plus an independent full-branch review closed hundreds of fidelity gaps across all three API dialects, every fix pinned by tests and the whole surface validated by real, unmodified client applications. Performance: hot paths now run in compiled native code and independent work in parallel, measurably cutting the gateway's processing overhead. Feature completeness: existing capabilities are rounded out end to end — explicit prompt caching, operator reasoning controls, the Responses API prompt parameter from Amazon Bedrock Prompt Management, native mid-conversation system messages on Claude 4.8+, richer speech and transcription (Polly speech marks, Transcribe/Translate extras, generic Converse speech-to-text), Cohere embedding_types and Rerank v1 structured documents, guardrail enforcement on every route, and an inline guardrail-checks moderation backend.

Behavior Changes

Review these before upgrading — they may change what existing clients observe:

Unsupported parameters are accepted and ignored, not rejected. Parameters the AWS backends cannot honour (e.g. known_speaker_*, partial_images, unsupported image quality/style, programmatic tool calling) now behave like they do on OpenAI: the request succeeds, the parameter is dropped, and a warning is recorded in the request log. Requests that returned 400 on v1.14 may now succeed.
Impossible combinations are now clean 400s instead of silent degradation: subtitle or diarized formats with stream=true, contradictory Amazon Transcribe settings, and web-search filters Nova grounding cannot apply are rejected with actionable messages.
Speech output quality: wav/flac/aac are now encoded from lossless PCM instead of Ogg Vorbis, and the default pcm output is resampled to 24 kHz for OpenAI parity (pass an explicit SampleRate to keep Polly's native rate). Same formats, different — better — bytes.
Error responses no longer expose backend internals. Server-side (5xx) error messages are generic with details kept in the server log, and Anthropic error types now match the official SDK exactly.
Comprehend-backed moderation always analyses text as English — the only language the AWS API accepts at runtime.
A configured guardrail now applies to every route. Embeddings, rerank, images, videos, and the audio routes enforce it through the ApplyGuardrail API (route coverage) — requests that silently bypassed the guardrail on v1.14 may now return 400 (code content_filter) or masked text, and each check is billed as guardrail text units.
SSRF protection covers every non-globally-reachable address. With SSRF_PROTECTION_BLOCK_PRIVATE_NETWORKS enabled (the default), a user-supplied URL resolving to shared address space (100.64.0.0/10, used by EKS custom networking and Hybrid Nodes) or another special-purpose range is now rejected with 403, alongside the RFC 1918 ranges.
Usage reporting is additive but richer: cached-token buckets are folded into prompt_tokens with prompt_tokens_details on every surface, and the Anthropic API now reports cache_creation_input_tokens (it was always null on v1.14).
Two request-body keys are reserved. model_id and additional_request_fields (plus stop_sequences on the legacy /v1/completions, where stop is the parameter to use) collide with the gateway's own request-building parameters: instead of being forwarded to Bedrock as provider extras, they return a 400 invalid_request_error naming the key.
The container health probe now respects TRUSTED_HOSTS. The image's HEALTHCHECK requests /health with a Host header derived from TRUSTED_HOSTS — a correct list keeps the container healthy with no extra entry. Deployments that re-declare the probe, such as an ECS task definition, should run the image's own command. Note that a load balancer health check still sends the target's IP address as the Host and is rejected with 400 when the allow-list is enabled.

Explicit Prompt Caching¶

Provider	Endpoint/Feature	AWS Backend
OpenAI	Explicit cache breakpoints on chat completions and responses, mapped to Bedrock `cachePoint` blocks with the per-request block budget enforced	Amazon Bedrock - Converse API
OpenAI	`prompt_cache_options` – cache TTL control on chat completions and responses, honoured on models that support it	Amazon Bedrock - Converse API

Prompt caching itself is not new — this release adds explicit placement and lifetime control on the OpenAI dialect, and fixes the caching plumbing that already existed (see Fixes).

Reasoning Controls¶

Provider	Endpoint/Feature	AWS Backend
stdapi.ai	`CHAT_COMPLETIONS_REASONING_FIELD` – return thinking text under `reasoning_content`, `reasoning`, or suppress it with `none`; applied identically to streamed deltas and final messages	Amazon Bedrock - Converse API & Mantle
OpenAI	OpenRouter-style `reasoning` request object accepted on chat completions (`effort`, `max_tokens`, `enabled`, `exclude`)	Amazon Bedrock - Converse API & Mantle

New API Features¶

Provider	Endpoint/Feature	AWS Backend
OpenAI	Responses API `prompt` parameter – serve prompts stored in Bedrock Prompt Management, with versions and variables (opt-in, `AWS_BEDROCK_ALLOW_PROMPT_ARN`)	Amazon Bedrock - Prompt Management
Anthropic / OpenAI	Mid-conversation system messages forwarded natively on Claude 4.8+ and Claude 5 family models instead of being folded into the system prompt	Amazon Bedrock - Converse API & Mantle
stdapi.ai	Guardrail asynchronous stream processing via the `X-Amzn-Bedrock-GuardrailStreamProcessingMode` request header	Amazon Bedrock - Guardrails
stdapi.ai	Configured guardrails enforced on every route: embeddings, rerank, images, videos, and audio now apply them via the ApplyGuardrail API (route coverage)	Amazon Bedrock - Guardrails
OpenAI	Moderations `amazon.bedrock-runtime-guardrail-checks` model – inline guardrail content filter checks with no guardrail resource required, the new default fallback for `omni-moderation-*` in supported regions	Amazon Bedrock - Guardrails