Python Client Libraries Integration¶

Build Python applications and agents directly on Amazon Bedrock models with stdapi.ai, using the same LangChain, pydantic-ai and OpenAI Agents SDK client classes you would use against OpenAI or Anthropic directly—three client-side changes: the base URL, the API key, and — where the name differs from what the client already sends — the model name, now picked from every provider in the catalogue rather than one vendor's list.

At a glance¶

%%{init: {'flowchart': {'htmlLabels': true}} }%%
flowchart LR
  app["<img src='../styles/logo_python.svg' style='height:64px;width:auto;vertical-align:middle;' /> Your Python App<br/>(LangChain, pydantic-ai, Agents SDK)"] --> stdapi["<img src='../styles/logo.svg' style='height:64px;width:auto;vertical-align:middle;' /> stdapi.ai"]
  stdapi --> bedrock["<img src='../styles/logo_amazon_bedrock.svg' style='height:64px;width:auto;vertical-align:middle;' /> Amazon Bedrock"]

About These Libraries¶

🔗 Links: LangChain | pydantic-ai | OpenAI Agents SDK | Ollama Python

These are among the most widely used Python libraries for building LLM-backed applications and agents. Every one of them accepts a custom base URL and API key as constructor arguments—no plugin, wrapper, or extension needed.

What you can build:

• Custom agents - Tool-calling loops, structured output, and multi-turn conversations in your own Python code
• RAG applications - Combine chat models with OpenAIEmbeddings for retrieval, see RAG Pipelines
• Stateful and voice agents - Server-side conversations, hosted retrieval and spoken sessions through the OpenAI Agents SDK
• Internal services - Backend applications and scripts that call Bedrock models without a UI or CLI in between
• Ollama-native code - Applications already written against the Ollama client, repointed at the gateway without changing their calls

Connect Your Own Instance¶

Point any Python process—wherever it runs—at your stdapi.ai gateway. Nothing below requires the AWS sample in Part 2.

Prerequisites¶

LangChain¶

Chat — langchain-openai¶

ChatOpenAI takes the gateway's /v1 base URL directly. .invoke(), .stream(), bind_tools(), and with_structured_output() all work unchanged against Bedrock models.

from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model="anthropic.claude-fable-5",
    base_url="https://YOUR_STDAPI_URL/v1",
    api_key="YOUR_STDAPI_KEY",
)

response = model.invoke("Name the largest planet in the solar system.")
print(response.content)

See Chat Completions API for the full parameter and model reference.

Embeddings — langchain-openai¶

OpenAIEmbeddings also takes the /v1 base URL, but its default behavior needs one extra setting.

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="amazon.titan-embed-text-v2:0",
    base_url="https://YOUR_STDAPI_URL/v1",
    api_key="YOUR_STDAPI_KEY",
    check_embedding_ctx_length=False,
)

vector = embeddings.embed_query("Your text here")

See Embeddings API for supported models.

Chat — langchain-anthropic¶

ChatAnthropic takes the gateway's /anthropic base URL and works with every model the route serves, not only Claude.

from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(
    model_name="anthropic.claude-fable-5",
    base_url="https://YOUR_STDAPI_URL/anthropic",
    api_key="YOUR_STDAPI_KEY",
)

response = model.invoke("Name the largest planet in the solar system.")
print(response.content)

See Anthropic Messages API for the full parameter and model reference.

pydantic-ai¶

pydantic-ai's OpenAIChatModel reaches the gateway through an OpenAIProvider carrying the base URL and API key, and works through the same Chat Completions API route as ChatOpenAI above—including reasoning models and multi-turn tool-calling loops.

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "anthropic.claude-fable-5",
    provider=OpenAIProvider(
        base_url="https://YOUR_STDAPI_URL/v1", api_key="YOUR_STDAPI_KEY"
    ),
)
agent = Agent(model, system_prompt="You are a helpful assistant.")

result = agent.run_sync("Name the largest planet in the solar system.")
print(result.output)

Reasoning-capable models (Claude, DeepSeek, and others) work through the same agent, including a full tool-calling loop that reasons on one turn and calls a registered tool on the next. Request a reasoning effort level per call with model_settings:

from pydantic_ai.models.openai import OpenAIChatModelSettings

result = agent.run_sync(
    "Call the registered tool, then answer.",
    model_settings=OpenAIChatModelSettings(openai_reasoning_effort="low"),
)

OpenAI Agents SDK¶

The OpenAI Agents SDK speaks the Responses API natively, so pointing its client at the gateway also hands its agents the server-side surfaces the Responses route serves — stored conversations, hosted retrieval and the realtime voice session included.

from agents import Agent, Runner, set_tracing_disabled
from agents.models.openai_responses import OpenAIResponsesModel
from openai import AsyncOpenAI

set_tracing_disabled(True)

client = AsyncOpenAI(base_url="https://YOUR_STDAPI_URL/v1", api_key="YOUR_STDAPI_KEY")
agent = Agent(
    name="assistant",
    instructions="Answer in one short sentence.",
    model=OpenAIResponsesModel(model="anthropic.claude-fable-5", openai_client=client),
)

result = Runner.run_sync(agent, "Name the largest planet in the solar system.")
print(result.final_output)

Server-Side Sessions¶

OpenAIConversationsSession keeps an agent's turns in a conversation on the gateway instead of in the process, so a second run replays what the gateway stored rather than a history you carried:

from agents.memory import OpenAIConversationsSession

session = OpenAIConversationsSession(openai_client=client)

Pass it as Runner.run_sync(..., session=session); the conversation id is what makes a run resumable from another process.

Hosted Retrieval¶

FileSearchTool(vector_store_ids=[...]) attaches a vector store to the agent, and the search happens inside the response — the SDK's own loop never sees a tool call. Build the store first, as in the RAG Pipelines guide, then:

from agents import FileSearchTool

agent = Agent(
    name="librarian",
    instructions="Answer only from the attached notes.",
    tools=[FileSearchTool(vector_store_ids=["vs_abc123"], include_search_results=True)],
)

Voice Agents¶

RealtimeRunner opens a spoken session against WS /v1/realtime: give its model_config the wss://YOUR_STDAPI_URL/v1/realtime?model=<id> URL and either the API key or a minted client secret. Sessions last up to 8 minutes and call no tools — see the Realtime API for what a session does and does not emit.

Ollama Python Client¶

The official ollama client reaches the gateway's Ollama-compatible endpoints. Point Client at your gateway and pass the API key as a bearer header — the same shape the client uses for any authenticated Ollama endpoint:

import ollama

client = ollama.Client(
    host="https://YOUR_STDAPI_URL/ollama",
    headers={"Authorization": "Bearer YOUR_STDAPI_KEY"},
)

response = client.chat(
    model="anthropic.claude-fable-5",
    messages=[
        {"role": "user", "content": "Name the largest planet in the solar system."}
    ],
)
print(response.message.content)

The host carries OLLAMA_ROUTES_PREFIX (/ollama by default): the client appends /api/chat, /api/tags and the rest itself. client.generate(), client.embed(), client.list(), client.show() and client.ps() all work against the same instance; client.pull() reports success immediately, because every model the gateway offers is hosted and available as soon as it appears in the model list.

See Ollama Chat API, Ollama Generate API, Ollama Embed API and Ollama Models API for the full parameter and model reference.

Deploy the Gateway on AWS¶

There is no application sample to deploy here — your LangChain, pydantic-ai, OpenAI Agents SDK or Ollama client process is code you already own. The Terraform sample below is one worked example of a credible AWS deployment for the gateway itself, not the only architecture that works: it is a normal HTTPS service, and where your Python process runs relative to its VPC is your choice, not a fixed part of the architecture.

Architecture¶

The decision that shapes this diagram is where your application process runs relative to the gateway's VPC: deployed as another ECS Fargate service (or any other workload) inside the same VPC, it reaches the gateway over private DNS and needs no public endpoint at all; running anywhere else — a laptop, a different account, another cloud — it reaches the gateway through a public Application Load Balancer instead.

%%{init: {'flowchart': {'htmlLabels': true, 'nodeSpacing': 20, 'rankSpacing': 40, 'subGraphTitleMargin': {'top': 8, 'bottom': 10}}} }%%
flowchart TB
  extapp["<img src='../styles/logo_python.svg' style='height:40px;width:auto;vertical-align:middle;' /> Your Python App<br/>(outside the VPC)"]

  subgraph public["Your VPC · public subnets"]
    alb["<img src='../styles/logo_amazon_load_balancing.svg' style='height:40px;width:auto;vertical-align:middle;' /> Application Load Balancer<br/>HTTPS · ACM certificate<br/>WAF (optional)"]
  end

  subgraph private["Your VPC · private app subnets — no inbound route from the internet"]
    intapp["<img src='../styles/logo_python.svg' style='height:40px;width:auto;vertical-align:middle;' /> Your Python App<br/>(in-VPC, e.g. ECS Fargate)"]
    stdapi["<img src='../styles/logo.svg' style='height:40px;width:auto;vertical-align:middle;' /> stdapi.ai<br/>ECS Fargate"]
    egress["<img src='../styles/logo_amazon_vpc.svg' style='height:40px;width:auto;vertical-align:middle;' /> NAT gateways · one per AZ<br/>+ free S3 gateway endpoint"]
  end

  subgraph regional["AWS service endpoints · your account, the regions you configure"]
    bedrock["<img src='../styles/logo_amazon_bedrock.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon Bedrock"]
    s3["<img src='../styles/logo_amazon_s3.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon S3<br/>SSE-KMS"]
    cw["<img src='../styles/logo_amazon_cloudwatch.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon CloudWatch<br/>logs · metrics · traces"]
  end

  extapp -->|"HTTPS · TLS 1.2+ · bearer token"| alb
  alb -->|"HTTP · private subnet"| stdapi
  alb ~~~ intapp
  intapp -.->|"OpenAI/Anthropic API · bearer token<br/>Cloud Map private DNS, no public endpoint"| stdapi
  stdapi --> egress
  egress -->|"HTTPS · SigV4"| bedrock
  egress -->|"S3 gateway endpoint"| s3
  egress --> cw

The solid path is the public one: an application outside the VPC has no route to the private subnets, so it can only reach the gateway through the ALB, over HTTPS. The dotted path is the in-VPC alternative: an application co-located in the private app subnets resolves the gateway through AWS Cloud Map private DNS and never touches the ALB — for that path, the ALB, its ACM certificate and any WAF module do not need to exist at all. A deployment picks one path or the other for a given application; both are shown here only because the choice is yours to make, not because both run at once.

What Each AWS Service Does Here¶

Security Measures in This Flow¶

• Authentication — every client class on this page sends its api_key argument as an Authorization: Bearer header, whichever mechanism validates it on the gateway: a stdapi.ai API key, an Amazon Cognito user-pool token the gateway verifies itself, or a token from an OIDC / IAM Identity Center flow terminated at the ALB before the request reaches stdapi.ai. AWS IAM SigV4 is available only through an API Gateway integration in front of the gateway — these SDKs' bearer-token clients cannot sign a SigV4 request themselves.
• Encryption in transit — HTTPS from an out-of-VPC application to the ALB; private-subnet HTTP from the ALB to the gateway container, or Cloud Map DNS with no ALB hop at all on the in-VPC path; HTTPS with SigV4 from the gateway to every AWS service it calls.
• Encryption at rest — SSE-KMS on the S3 bucket that holds multimodal payloads, with a customer-managed key.
• Least privilege — the gateway's ECS task role carries only the model and AI-service actions its configuration enables, not a blanket Bedrock or S3 grant.
• Content policy — a Bedrock guardrail configured on the gateway applies to every chat request your application sends, independent of which client library issued it.
• Data handling — the gateway is stateless and holds request bodies in memory only; no third party sits between your application and the models it calls, so a Bedrock request made through stdapi.ai carries no telemetry back to another vendor unless your own client library adds it — see the set_tracing_disabled(True) note under OpenAI Agents SDK above.

What It Costs to Run¶

Read a model's price before your application sends anything to it with GET /model_pricing. Setting COST_TRACKING=true additionally puts a per-request cost on each usage entry — estimated from published AWS prices, not read back from your invoice.

What to Watch¶

The gateway writes one structured request event per call to CloudWatch, carrying the request id, path, status code, execution_time_ms, the model that served it, and the token counts AWS billed; streaming calls add a matching request_stream event. When OTEL_ENABLED=true and OTEL_EXPORTER_ENDPOINT point at a collector, the gateway also exports OpenTelemetry traces for the same calls. Because your application is the client here, the most direct link between the two is the x-request-id response header: read it from every gateway response and log it alongside your own request handling, so a failure your application sees can be traced straight back to the gateway event that produced it.

fields id, path, model_id, status_code
| filter type = "request" and status_code >= 400
| stats count(*) as errors by path, model_id, status_code
| sort errors desc

Pair this with your application's own logs, keyed on the x-request-id it received, to tell a gateway-side failure apart from one your code introduced before or after the call.

Next Steps¶