Compare stdapi.ai¶

Full Bedrock catalog supported; each model must be declared in config (applies to auto-discovery) ↩↩

Subset of Bedrock models — mostly newer open-weight models (DeepSeek, Gemma, Qwen, GLM, MiniMax, Kimi, newer Mistral) plus OpenAI GPT/gpt-oss and xAI Grok; newer Anthropic Claude generations are available via the Messages API only. Amazon Nova, Meta Llama, AI21, Cohere, and Stability AI (images) are not available — see AWS endpoint availability ↩↩↩↩

CloudFormation reference sample (API Gateway + Lambda, or ALB + Fargate) — no WAF, auto-scaling, monitoring, or commercial support ↩

Covered through your existing AWS Support plan ↩

Generalist multi-cloud proxy covering 100+ providers; AWS-specific Bedrock features and security integrations may lag behind dedicated solutions ↩

Prompt caching and reasoning supported on standard routes; coverage varies by model — not all Bedrock models support prompt caching or extended thinking ↩

Auto-discovery limited to the single deployed region — some models are only available in specific AWS regions ↩

Achievable via the LiteLLM router, but requires manually declaring each model per region with explicit TPM/RPM limits — no automatic quota distribution ↩

Prompt caching on Mantle is supported but model-dependent (per each model card); reasoning is served via frontier models (GPT-5.x, gpt-oss) and select open-weight models (Qwen3 thinking, etc.) ↩

Single-region deployment — some models are only available in specific AWS regions; no cross-region catalog aggregation ↩

100+ providers: OpenAI, Azure OpenAI, GCP Vertex, Anthropic direct, and more — ideal when you need a single gateway across multiple clouds ↩

Requires connecting Amazon Polly as the TTS backend — not included by default in a LiteLLM on AWS deployment ↩

Supported by specifying the cross-region inference profile ARN as the model ID — no automatic profile selection ↩↩

Anthropic Messages API supported on Mantle for newer Claude generations only — Claude 3.x and some 4.x are not available; structured outputs (output_config.format) are rejected with a 400 on Mantle — see AWS endpoint availability ↩

LiteLLM ships a first-class bedrock_mantle/ provider covering all Mantle models, so reaching Mantle needs no custom endpoint. Bedrock and Mantle remain two separate providers, each model declared in model_list, rather than one auto-discovered catalog ↩

LiteLLM translates between API formats per provider adapter; coverage is not a guaranteed all-text-APIs-on-every-model surface ↩

Conversation state kept in LiteLLM's own datastore rather than native provider server-side storage ↩

Served via LiteLLM's /responses endpoint, auto-bridged to /chat/completions; no native Bedrock Responses passthrough ↩

Mantle uses Projects/Workspaces (tag-based cost allocation via Cost Explorer) for workload isolation instead of application inference profiles — see Amazon Bedrock Projects ↩

LiteLLM provides an MCP gateway that proxies configured third-party MCP servers and converts OpenAPI specs to tools — a related capability, but it does not expose LiteLLM's own AI and media endpoints as tools ↩

Amazon Bedrock is a first-class batch provider for LiteLLM's Batches API — on the proxy only, not the SDK, and backed by Managed Files with a Postgres database plus an operator-configured S3 bucket and Bedrock batch role. LiteLLM's own cost tracking for batches is an Enterprise feature ↩

The Bedrock Mantle endpoint serves the Responses, Chat Completions and Anthropic Messages APIs only. Amazon Bedrock does offer an OpenAI-compatible Batch API on its bedrock-runtime endpoint — for OpenAI models only, reading its input from an S3 URI you upload yourself rather than through a Files API ↩

With Amazon Bedrock it is a bridge to a knowledge base you registered yourself: search, retrieve, list, update and delete are forwarded, while OpenAI-shaped store creation and file attachment accept the openai provider only. LiteLLM's own /v1/rag/ingest route does chunk, embed and index into a Bedrock knowledge base, but it is not part of the OpenAI API surface an unmodified client speaks ↩

Emulated: the store is presented to the model as a function tool and the search is executed by the proxy rather than by the model's own server. LiteLLM documents that file_citation annotations can come back empty when the vector backend does not return the file metadata they need ↩

Spend is attributed per key and per user in LiteLLM's own database, not in AWS's billing data. Bedrock requestMetadata can be forwarded, which AWS surfaces in invocation logs rather than Cost Explorer or CUR 2.0 ↩

Application inference profiles put billed dollars in Cost Explorer and CUR 2.0, but per application rather than per end user — the gateway does not map a caller to a profile ↩

IAM principal attribution, Projects and Workspaces put billed dollars in Cost Explorer and CUR 2.0 — per identity or per project rather than per end user, and behind a shared gateway every caller arrives as the same identity ↩

/images/edits lists AWS Bedrock (Stability) among its providers, so Stability's editing operations are reachable on Bedrock, and the Bedrock image generation page now documents image_edit() against Amazon Nova Canvas, mapping to the IMAGE_VARIATION, INPAINTING, OUTPAINTING and BACKGROUND_REMOVAL task types. Amazon Titan remains documented for generation only ↩

LiteLLM integrates Ollama as one of the backends it calls. Its proxy does not serve Ollama's own /api/* routes, so an Ollama-native client cannot point at it ↩

Reachable by declaring the endpoint's ARN as a model entry (bedrock/<provider_route>/<arn>) in the Bedrock provider configuration: one entry per endpoint, and no discovery of what is deployed ↩

The model list is discovered from Bedrock's foundation models and inference profiles only. The project's README documents Marketplace, custom-model and provisioned-throughput ARNs as unsupported, and states there is no plan to support SageMaker models ↩↩

A model deployed to a Marketplace or SageMaker AI endpoint is invoked through Bedrock's own Converse and InvokeModel APIs, and is absent from Mantle's API compatibility matrix ↩↩

Each endpoint is declared in model_list through the sagemaker_chat provider. Endpoints are not discovered, and nothing is documented for an endpoint scaled to zero ↩

A single API key is read from AWS Secrets Manager, Parameter Store or the environment at start-up, so there is no per-caller key to scope ↩

Bedrock API keys are restricted by an IAM policy on the identity behind the key rather than by a model or endpoint scope carried on the key itself ↩

The same data is served by LiteLLM's own /spend/* routes, reported per key, user and team in a different wire format, so a dashboard written against OpenAI's Administration API does not read it ↩

Deprecated and archived by AWS on 4 September 2026: "This project is deprecated. Amazon Bedrock now serves OpenAI-compatible and Anthropic-compatible APIs natively, which is the reason this proxy existed. Call Amazon Bedrock directly instead of deploying this gateway." AWS's stated migration path is the bedrock-mantle endpoint, the rightmost column. The column is kept because existing deployments still run, and because it remains the best-documented account of what a minimal Bedrock proxy covers ↩

Amazon Bedrock is an AWS service: inference runs under your own AWS account, IAM and bill, and no third party sits in the path. What bedrock-mantle is not is self-hosted, so you do not pin its version or run it in your own VPC; see the Self-hosted row. ↩

/v1/files accepts an Amazon Bedrock batch input file, on the proxy only, with purpose=batch and target_model_names naming the batch model. The file is written to an S3 bucket the operator configured, and the results are read back through files.content with a custom-llm-provider: bedrock header or an s3:// URI (Bedrock Batches). It rides Managed Files, which is beta and requires a PostgreSQL database. Batch input is the only documented purpose for Bedrock, and there is no OpenAI Uploads API, so a file too large for a single request has nowhere to go ↩

This column scores the bedrock-mantle endpoint. AWS serves the same OpenAI-compatible Chat Completions and Responses APIs and the Anthropic Messages API on a second endpoint, bedrock-runtime, under its /openai/v1 paths, and recommends that one for most new applications. It is the same inference engine at identical per-token pricing, and it adds Amazon Bedrock Guardrails, intelligent prompt routing, geographic and global cross-region inference, and the Bedrock-native Converse and InvokeModel APIs. What only bedrock-mantle carries is background inference, server-side and pre-configured tools, Projects and Workspaces, and the models bedrock-runtime does not serve at all, among them GPT-5.4, GPT-5.5, both Daybreak variants and Grok 4.3 ↩↩↩

Mantle runs server-side and pre-configured tools of its own, web search among them, which the bedrock-runtime endpoint does not. They are not the Bedrock system tools a model reaches through the Converse API, and bedrock-mantle does not serve Converse ↩