Compare stdapi.ai¶

Full Bedrock catalog supported; each model must be declared in config (applies to auto-discovery) ↩↩

Subset of Bedrock models — mostly newer open-weight models (DeepSeek, Gemma, Qwen, GLM, MiniMax, Kimi, newer Mistral) plus OpenAI GPT/gpt-oss and xAI Grok; newer Anthropic Claude generations are available via the Messages API only. Amazon Nova, Meta Llama, AI21, Cohere, and Stability AI (images) are not available — see AWS endpoint availability ↩↩↩↩

CloudFormation reference sample (API Gateway + Lambda, or ALB + Fargate) — no WAF, auto-scaling, monitoring, or commercial support ↩

Covered through your existing AWS Support plan ↩

Generalist multi-cloud proxy covering 100+ providers; AWS-specific Bedrock features and security integrations may lag behind dedicated solutions ↩

Prompt caching and reasoning supported on standard routes; coverage varies by model — not all Bedrock models support prompt caching or extended thinking ↩

Auto-discovery limited to the single deployed region — some models are only available in specific AWS regions ↩

Achievable via the LiteLLM router, but requires manually declaring each model per region with explicit TPM/RPM limits — no automatic quota distribution ↩

Prompt caching on Mantle is supported but model-dependent (per each model card); reasoning is served via frontier models (GPT-5.x, gpt-oss) and select open-weight models (Qwen3 thinking, etc.) ↩

Single-region deployment — some models are only available in specific AWS regions; no cross-region catalog aggregation ↩

100+ providers: OpenAI, Azure OpenAI, GCP Vertex, Anthropic direct, and more — ideal when you need a single gateway across multiple clouds ↩

Requires connecting Amazon Polly as the TTS backend — not included by default in a LiteLLM on AWS deployment ↩

Supported by specifying the cross-region inference profile ARN as the model ID — no automatic profile selection ↩↩

Anthropic Messages API supported on Mantle for newer Claude generations only — Claude 3.x and some 4.x are not available; structured outputs (output_config.format) are rejected with a 400 on Mantle — see AWS endpoint availability ↩

LiteLLM ships a first-class bedrock_mantle/ provider covering all Mantle models, so reaching Mantle needs no custom endpoint. Bedrock and Mantle remain two separate providers, each model declared in model_list, rather than one auto-discovered catalog ↩

LiteLLM translates between API formats per provider adapter; coverage is not a guaranteed all-text-APIs-on-every-model surface ↩

Conversation state kept in LiteLLM's own datastore rather than native provider server-side storage ↩

Served via LiteLLM's /responses endpoint, auto-bridged to /chat/completions; no native Bedrock Responses passthrough ↩

Mantle uses Projects/Workspaces (tag-based cost allocation via Cost Explorer) for workload isolation instead of application inference profiles — see Amazon Bedrock Projects ↩

LiteLLM provides an MCP gateway that proxies configured third-party MCP servers and converts OpenAPI specs to tools — a related capability, but it does not expose LiteLLM's own AI and media endpoints as tools ↩

Amazon Bedrock is a first-class batch provider for LiteLLM's Batches API — on the proxy only, not the SDK, and backed by Managed Files with a Postgres database plus an operator-configured S3 bucket and Bedrock batch role. LiteLLM's own cost tracking for batches is an Enterprise feature ↩

The Bedrock Mantle endpoint serves the Responses, Chat Completions and Anthropic Messages APIs only. Amazon Bedrock does offer an OpenAI-compatible Batch API on its bedrock-runtime endpoint — for OpenAI models only, reading its input from an S3 URI you upload yourself rather than through a Files API ↩

With Amazon Bedrock it is a bridge to a knowledge base you registered yourself: search, retrieve, list, update and delete are forwarded, while OpenAI-shaped store creation and file attachment accept the openai provider only. LiteLLM's own /v1/rag/ingest route does chunk, embed and index into a Bedrock knowledge base, but it is not part of the OpenAI API surface an unmodified client speaks ↩

Emulated: the store is presented to the model as a function tool and the search is executed by the proxy rather than by the model's own server. LiteLLM documents that file_citation annotations can come back empty when the vector backend does not return the file metadata they need ↩

Spend is attributed per key and per user in LiteLLM's own database, not in AWS's billing data. Bedrock requestMetadata can be forwarded, which AWS surfaces in invocation logs rather than Cost Explorer or CUR 2.0 ↩

Application inference profiles put billed dollars in Cost Explorer and CUR 2.0, but per application rather than per end user — the gateway does not map a caller to a profile ↩

IAM principal attribution, Projects and Workspaces put billed dollars in Cost Explorer and CUR 2.0 — per identity or per project rather than per end user, and behind a shared gateway every caller arrives as the same identity ↩

/images/edits lists AWS Bedrock (Stability) among its providers, so Stability's editing operations are reachable on Bedrock. It does not extend to the other Bedrock image models: the Bedrock provider page lists /images/generations but not /images/edits among its supported OpenAI endpoints, and Amazon Nova Canvas and Titan are documented for generation only ↩

LiteLLM integrates Ollama as one of the backends it calls. Its proxy does not serve Ollama's own /api/* routes, so an Ollama-native client cannot point at it ↩

Reachable by declaring the endpoint's ARN as a model entry (bedrock/<provider_route>/<arn>) in the Bedrock provider configuration: one entry per endpoint, and no discovery of what is deployed ↩

The model list is discovered from Bedrock's foundation models and inference profiles only. The project's README documents Marketplace, custom-model and provisioned-throughput ARNs as unsupported, and states there is no plan to support SageMaker models ↩↩

A model deployed to a Marketplace or SageMaker AI endpoint is invoked through Bedrock's own Converse and InvokeModel APIs, and is absent from Mantle's API compatibility matrix ↩↩

Each endpoint is declared in model_list through the sagemaker_chat provider. Endpoints are not discovered, and nothing is documented for an endpoint scaled to zero ↩

A single API key is read from AWS Secrets Manager, Parameter Store or the environment at start-up, so there is no per-caller key to scope ↩

Bedrock API keys are restricted by an IAM policy on the identity behind the key rather than by a model or endpoint scope carried on the key itself ↩

The same data is served by LiteLLM's own /spend/* routes, reported per key, user and team in a different wire format, so a dashboard written against OpenAI's Administration API does not read it ↩