Configuration Guide¶

stdapi.ai is configured entirely through environment variables, which are read once at startup and cannot be changed without restarting the service. This guide explains each setting category with practical examples to help you configure the service correctly.

What you can configure:

• AWS regions - Access models across multiple regions for availability and model selection
• Data sovereignty - Control which AWS regions are used for compliance (GDPR, HIPAA, etc.)
• Storage - S3 buckets for file operations, regional buckets for multi-region deployments
• Authentication - API keys via SSM or Secrets Manager for secure access control
• Observability - Logging levels, OpenTelemetry, request/response debugging
• Security - CORS, proxy headers, trusted hosts for production deployments
• Performance - Caching, model overrides, S3 acceleration
• TLS / SSL - End-to-end encryption using Granian environment variables

Quick Start¶

For production deployments, configure these essential settings:

Minimal Production Setup¶

Single-region deployment with file storage only.

# S3 bucket for file storage (must be in same region as your server)
export AWS_S3_BUCKET=my-stdapi-bucket

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Production with Authentication¶

Adds secure API key authentication via AWS Systems Manager.

# S3 bucket for file storage (must be in same region as your server)
export AWS_S3_BUCKET=my-stdapi-bucket

# Secure API authentication (recommended: SSM Parameter Store)
export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Full Production Setup (All Features Enabled)¶

Multi-region deployment with all AWS AI services, observability, and security features.

# Core AWS configuration - host server in first region
export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,eu-west-1

# S3 bucket for file storage (must be in us-east-1, your first/primary region)
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket

# Optional: Transcribe S3 bucket (defaults to AWS_S3_BUCKET if not specified)
# Only set this if you need a separate bucket or if transcribe is in a different region
# export AWS_TRANSCRIBE_S3_BUCKET=my-stdapi-transcribe-us-east-1

# Optional: Regional buckets for async/batch inference in other regions
export AWS_S3_REGIONAL_BUCKETS='{"us-west-2": "my-stdapi-us-west-2-bucket", "eu-west-1": "my-stdapi-eu-west-1-bucket"}'

# AWS AI services regions (optional - when unset, every AWS_BEDROCK_REGIONS entry is a
# candidate with automatic failover; set one to pin the service to a single region)
export AWS_POLLY_REGION=us-east-1           # Text-to-speech
export AWS_TRANSCRIBE_REGION=us-east-1      # Speech-to-text (audio transcription)
export AWS_COMPREHEND_REGION=us-east-1      # Language detection & moderation
export AWS_TRANSLATE_REGION=us-east-1       # Text translation

# Authentication
export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

# Logging
export LOG_LEVEL=warning
export LOG_CLIENT_IP=true

# Optional: OpenTelemetry observability (AWS X-Ray integration)
# export OTEL_ENABLED=true
# export OTEL_SERVICE_NAME=stdapi-production
# export OTEL_SAMPLE_RATE=0.1

# Production security settings (when behind AWS ALB/CloudFront)
export ENABLE_PROXY_HEADERS=true

# Note: TRUSTED_HOSTS not recommended with AWS ALB - use ALB host-based routing instead
# Only use TRUSTED_HOSTS if you cannot configure host validation at the load balancer level

# Optional: CORS for browser-based web applications
# export CORS_ALLOW_ORIGINS='["https://app.example.com"]'

Development Setup¶

Local development configuration with API documentation and debug logging enabled.

# Minimal configuration for local development
export AWS_S3_BUCKET=my-stdapi-dev-bucket

# Enable API documentation
export ENABLE_DOCS=true
export ENABLE_REDOC=true

# Full request/response logging for debugging
export LOG_LEVEL=info
export LOG_REQUEST_PARAMS=true

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Environment Variable Summary¶

This section provides a quick reference of all available configuration options. Detailed explanations for each variable can be found in the sections below.

Essential (Production)¶

AWS Client¶

AWS Storage¶

AWS AI Services¶

Resilience & Failover¶

Bedrock Mantle¶

Bedrock Advanced¶

Authentication¶

Configure one API key source. If several are set, precedence is API_KEY → SSM Parameter Store → Secrets Manager — see Authentication:

Amazon Cognito user pool tokens are an alternative to the API key — see Amazon Cognito Authentication:

Publishing where tokens come from lets an AI agent authenticate itself — see Authentication Discovery:

API Compatibility¶

Logging¶

CloudWatch Metrics¶

Cost Tracking¶

Observability (OpenTelemetry)¶

HTTP/Security¶

Application Behavior¶

API Documentation¶

MCP (Model Context Protocol)¶

AWS Services and Regions¶

General Configuration¶

AWS_ADAPTIVE_RETRY¶

Purpose : Enable adaptive retry mode that adjusts retry pacing based on observed error rates across all AWS service calls

Type : Boolean (true / false)

Default : false

Behavior : When enabled, the retry strategy dynamically responds to real-time congestion signals. If errors are occurring frequently, retries are spaced further apart to avoid amplifying load on an already-stressed endpoint. Once conditions improve, the pacing returns to normal. When disabled, retries follow a standard exponential backoff strategy with fixed intervals. Applies to all AWS services (Bedrock, S3, Polly, Transcribe, etc.).

# Default: standard exponential backoff
export AWS_ADAPTIVE_RETRY=false

# Enable adaptive retry (recommended under sustained high load)
export AWS_ADAPTIVE_RETRY=true

AWS_MAX_POOL_CONNECTIONS¶

Purpose : Maximum number of concurrent HTTP connections per AWS service client

Type : Integer (must be > 0)

Default : 50

Behavior : Each AWS service client (one per service per region) maintains its own connection pool up to this limit. Under high concurrency, increasing this value prevents requests from queuing for an available connection. Setting it too high may exhaust system file descriptors.

# Default
export AWS_MAX_POOL_CONNECTIONS=50

# High-concurrency deployment
export AWS_MAX_POOL_CONNECTIONS=100

AWS_CONNECT_TIMEOUT¶

Purpose : Timeout in seconds for establishing a connection to an AWS service endpoint

Type : Integer (must be > 0)

Default : 5

Behavior : Limits how long the client waits when opening a new connection. It also bounds, per candidate region, the time a real-time audio session (generative-voice speech, speech-to-speech) may take to become ready: connection, initial handshake and the first response together. A short value allows fast failover to another region when an endpoint is unreachable. Increase it only if you see spurious connection timeouts on high-latency networks, or real-time audio requests failing with a 503 a few seconds after they start.

# Default: 5 seconds
export AWS_CONNECT_TIMEOUT=5

# High-latency network
export AWS_CONNECT_TIMEOUT=10

Storage Configuration¶

AWS_S3_BUCKET¶

Purpose : Primary S3 bucket for storing generated files (images, audio, documents) and temporary data during processing

Default : None (must be configured for file operations)

Best Practice : The bucket must be in the first region specified in AWS_BEDROCK_REGIONS (your primary region where the server should be hosted) to avoid cross-region data transfer costs and reduce latency

export AWS_S3_BUCKET=my-llm-storage-us-east-1

AWS_S3_ACCELERATE¶

Purpose : Enable S3 Transfer Acceleration for presigned URLs to improve download performance for large files

Type : Boolean

Default : false

Best Practice : Enable when serving large files (high-resolution images, audio) to geographically distributed users

export AWS_S3_ACCELERATE=true

AWS_S3_TMP_PREFIX¶

Purpose : S3 prefix (folder path) for temporary files used during job processing

Default : tmp/

Best Practice : Configure S3 lifecycle policies to automatically delete objects under this prefix after 1 day

export AWS_S3_TMP_PREFIX=tmp/

Custom prefix examples:

# Production environment
export AWS_S3_TMP_PREFIX=prod/tmp/

# Staging environment
export AWS_S3_TMP_PREFIX=staging/tmp/

# Organize by date (requires manual updates)
export AWS_S3_TMP_PREFIX=tmp/2025/01/

# No prefix (store at bucket root - not recommended)
export AWS_S3_TMP_PREFIX=

AWS_S3_FILES_PREFIX¶

Purpose : S3 prefix (folder path) for Files API objects (OpenAI and Anthropic /v1/files endpoints)

Default : files/

Best Practice : Configure an AbortIncompleteMultipartUpload S3 lifecycle rule on this prefix to clean up abandoned upload parts, and apply Intelligent-Tiering for cost optimisation

export AWS_S3_FILES_PREFIX=files/

Custom prefix examples:

# Production environment
export AWS_S3_FILES_PREFIX=prod/files/

# Staging environment
export AWS_S3_FILES_PREFIX=staging/files/

# No prefix (store at bucket root - not recommended)
export AWS_S3_FILES_PREFIX=

AWS_S3_VIDEOS_PREFIX¶

Purpose : S3 prefix (folder path) for videos generated through the Videos API

Default : videos/

Requirement : Must be non-empty, use only S3-safe characters (alphanumerics plus ! _ . * ' ( ) - per path segment), and end with a trailing / — an empty value would widen the ownership check that scopes listing/retrieval to the whole bucket

Best Practice : Generated videos persist until deleted through the API — configure an S3 lifecycle rule on this prefix to cap storage costs

export AWS_S3_VIDEOS_PREFIX=videos/

Amazon Bedrock writes each video generation job's output (MP4 and manifest) under this prefix, in a folder named after the job. Because Amazon Bedrock requires the output bucket to be in the same region as the invocation, videos are stored in the AWS_S3_REGIONAL_BUCKETS bucket of the region that served the job.

AWS_S3_VIDEOS_EXPIRES_AFTER¶

Purpose : Retention period in seconds (minimum 3600) for videos generated through the Videos API

Default : Unset — videos never expire and persist until deleted through the API

Best Practice : Pair with an S3 Lifecycle expiration rule on AWS_S3_VIDEOS_PREFIX covering the same duration (rounded up to whole days) so the objects are actually deleted

# Expire generated videos after 24 hours
export AWS_S3_VIDEOS_EXPIRES_AFTER=86400

When set, the Video object reports expires_at (job completion time plus this value) and downloading expired video content returns a 404. The server enforces expiry at the API level only; the paired S3 Lifecycle rule performs the physical cleanup.

AWS_S3_BATCHES_PREFIX¶

Purpose : S3 prefix (folder path) for the data of the Batch API and the Message Batches API — the submitted requests, the results, and the batch records themselves

Default : batches/

Requirement : Must be non-empty, use only S3-safe characters (alphanumerics plus ! _ . * ' ( ) - per path segment), and end with a trailing / — batches are addressed by prefix, so an empty value would make every stray object at the bucket root a candidate

Best Practice : Batch data persists until the batch is deleted — configure an S3 lifecycle rule on this prefix to cap storage costs, and grant AWS_BEDROCK_BATCH_ROLE_ARN read and write access under it

export AWS_S3_BATCHES_PREFIX=batches/

Each batch stores its data under a folder of its own below this prefix. Because the output bucket must be in the same region as the model that serves the batch, that data is stored in the AWS_S3_REGIONAL_BUCKETS bucket of the region that served it.

Vector Stores¶

The Vector Stores API needs an Amazon S3 vector bucket — a resource type of its own, created separately from a general purpose bucket — plus the general purpose bucket in AWS_S3_BUCKET, which holds the stores' records. The vector store endpoints answer 503 until both are configured, or until AWS_BEDROCK_KNOWLEDGE_BASE_IDS names a knowledge base to serve instead.

AWS_S3_VECTORS_BUCKET¶

Purpose : Name of the Amazon S3 vector bucket that holds the indexed content of every vector store

Default : None — the Vector Stores API is disabled

Requirement : Requires AWS_S3_BUCKET, which holds the vector store records; startup fails if only the vector bucket is set. The gateway's role needs the Vector Stores permissions on it

export AWS_S3_VECTORS_BUCKET=my-llm-vectors-us-east-1

Create the bucket yourself, then let the gateway create and delete the indexes inside it — one per vector store, removed when the store is deleted or expires.

AWS_S3_VECTORS_REGION¶

Purpose : AWS region holding AWS_S3_VECTORS_BUCKET

Default : The first AWS_BEDROCK_REGIONS entry

Requirement : Must be the bucket's own region. A vector bucket is a regional resource whose content is only reachable there, so this setting has no failover: if the region is unreachable, so is the Vector Stores API

export AWS_S3_VECTORS_REGION=us-east-1

AWS_S3_VECTOR_STORES_PREFIX¶

Purpose : S3 prefix (folder path) in AWS_S3_BUCKET for the Vector Stores API records — the stores, their attached files and their file batches

Default : vector_stores/

Best Practice : Keep it distinct from the other prefixes so a lifecycle rule written for one never reaches the records of another; these records are the stores themselves, so no expiration rule belongs on this prefix

export AWS_S3_VECTOR_STORES_PREFIX=vector_stores/

VECTOR_STORE_EMBEDDING_MODEL¶

Purpose : Model that turns the indexed files and the search queries into vectors

Default : amazon.titan-embed-text-v2:0

Requirement : Must be an embedding model available in your configured regions — see Embeddings API

export VECTOR_STORE_EMBEDDING_MODEL=amazon.titan-embed-text-v2:0

Each vector store records the model it was created with and keeps using it, so changing this setting only affects stores created afterwards. Existing stores keep answering exactly as before.

VECTOR_STORE_CHUNK_SIZE_TOKENS¶

Purpose : Default chunk size, in tokens, for files indexed without an explicit chunking_strategy

Default : 800 — the same default the upstream API applies

Requirement : Between 100 and 4096

export VECTOR_STORE_CHUNK_SIZE_TOKENS=800

A request's own chunking_strategy always wins, and a store created with one applies it to every file later attached without one. Chunk sizes are approximate — see Chunking.

VECTOR_STORE_CHUNK_OVERLAP_TOKENS¶

Purpose : Default number of tokens shared between consecutive chunks, for files indexed without an explicit chunking_strategy

Default : 400 — the same default the upstream API applies

Requirement : Must not exceed half of VECTOR_STORE_CHUNK_SIZE_TOKENS; startup fails otherwise

export VECTOR_STORE_CHUNK_OVERLAP_TOKENS=400

More overlap keeps a sentence split across two chunks findable from either one, at the cost of more chunks to embed and store.

AWS_SQS_VECTOR_STORE_QUEUE_URL¶

Purpose : URL of the Amazon SQS standard queue that carries vector store indexing work, so a file keeps being indexed when the server that accepted it stops

Default : None — indexing runs in the server that accepted the request, and a file being indexed when that server stops is reported as failed

Requirement : Requires AWS_S3_VECTORS_BUCKET. Must be a standard queue (not FIFO), with a dead-letter queue, and the gateway's role needs the Durable vector store indexing permissions on it

export AWS_SQS_VECTOR_STORE_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/stdapi-ai-indexing

Attaching a file records the work on the queue before answering, and every server reads from that queue: whichever one is still running finishes it. The region comes from the URL, so there is nothing else to configure. Create the queue yourself, in the same account.

The queue only ever carries identifiers — which store, which files, which batch — never file content, so no indexed data is stored a second time.

See Durable indexing for what a client observes, and Resilience for how it behaves during a deployment.

AWS_BEDROCK_KNOWLEDGE_BASE_IDS¶

Purpose : Comma-separated allowlist of Amazon Bedrock knowledge bases served through the Vector Stores API. Each allowlisted knowledge base is addressed as the vector store vs_kb_<knowledgeBaseId> on every /v1/vector_stores endpoint, and is returned by GET /v1/vector_stores next to the stores the server owns

Default : Empty — no knowledge base is addressable, and a vs_kb_... identifier is answered exactly as an unknown vector store is, so the allowlist cannot be probed

Requirement : Each knowledge base must already exist, in the first AWS_BEDROCK_REGIONS entry, be a Bedrock managed (MANAGED) or customer-managed (VECTOR) knowledge base, and the gateway's role needs the Knowledge Base Vector Stores permissions on it

export AWS_BEDROCK_KNOWLEDGE_BASE_IDS=ABCDE12345,FGHIJ67890/KLMNO13579

Write each entry as <knowledgeBaseId>, or as <knowledgeBaseId>/<dataSourceId> when the knowledge base has more than one data source; with a single data source the server resolves it itself.

Both kinds of document knowledge base are served, and a store behaves the same on either. A knowledge base connected to a structured data store (SQL) or backed by an Amazon Kendra GenAI index (KENDRA) is not a vector store and must not be allowlisted.

The knowledge base itself always stays yours: the server never creates one and never deletes one. It searches it, and manages the documents of its data source. Name, description, creation time and status are read from the knowledge base, and the requests that would change them are refused — see Knowledge Base Stores.

This setting is independent of AWS_S3_VECTORS_BUCKET: a deployment that sets only this one serves its allowlisted knowledge bases and creates no store of its own.

AWS_TRANSCRIBE_S3_BUCKET¶

Purpose : Temporary S3 bucket for transcription workflows

Default : Falls back to AWS_S3_BUCKET if not specified

Requirement : Must be in the same region as AWS_TRANSCRIBE_REGION when that is set; with the default multi-region behavior it serves the primary Bedrock region, and AWS_S3_REGIONAL_BUCKETS entries serve the other candidate regions

# If AWS_TRANSCRIBE_REGION is us-east-1
export AWS_TRANSCRIBE_S3_BUCKET=my-transcribe-temp-us-east-1

# If AWS_TRANSCRIBE_REGION is eu-west-1
export AWS_TRANSCRIBE_S3_BUCKET=my-transcribe-temp-eu-west-1

AWS_TRANSCRIBE_STREAM_LANGUAGES¶

Purpose : Languages a streamed transcription (stream=true) picks between when the request names none

Default : Empty — a request naming no language is transcribed once the whole recording has been read, and its language detected

Format : JSON array of two or more language codes; a single entry has no effect

A streamed transcription returns text before the recording has been fully read, which requires knowing which languages to expect. A request that names its language — or two or more expected languages — always gets one. Set this to extend the same behavior to requests that name neither, listing the languages your callers actually send. Listing more than five is not recommended, and two variants of the same language (en-US and en-GB) cannot both appear.

export AWS_TRANSCRIBE_STREAM_LANGUAGES='["en-US", "es-US", "fr-FR"]'

AWS_TRANSCRIBE_OUTPUT_ENCRYPTION_KEY_ARN¶

Purpose : AWS KMS key encrypting the transcription output written to AWS_TRANSCRIBE_S3_BUCKET

Default : None — output objects keep the bucket's own default encryption (SSE-S3)

Format : KMS key ARN: arn:<partition>:kms:<region>:<account-id>:key/<key-id>; startup fails on any other value

Requirement : The server's role needs kms:GenerateDataKey and kms:Decrypt on the key (IAM Permissions). With the default multi-region behavior the key must be usable from every candidate region — a multi-Region key or a single AWS_TRANSCRIBE_REGION

export AWS_TRANSCRIBE_OUTPUT_ENCRYPTION_KEY_ARN=arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012

Each job also sends its request identifiers (stdapi-ai.request_id, stdapi-ai.server_id, and stdapi-ai.user_id when the user is known) as the KMS encryption context, so a key policy can be conditioned on them.

AWS_S3_REGIONAL_BUCKETS¶

Purpose : Region-specific S3 buckets for Bedrock async and batch inference operations, and for staging attachments too large to travel inside a request

Default : Empty (no regional buckets configured)

Format : JSON object with region names as keys and bucket names as values

Requirement : Some Bedrock models require S3 buckets in the same region for async and batch inference operations

export AWS_S3_REGIONAL_BUCKETS='{"us-east-1": "my-bedrock-temp-us-east-1", "eu-west-1": "my-bedrock-temp-eu-west-1"}'

AWS_S3_ACCEPTED_BUCKETS¶

Purpose : Declare external S3 buckets that the application has read access to, mapped to their AWS region

Type : JSON object (keys: bucket names, values: AWS region identifiers)

Default : {} (empty — only the application's own buckets are recognized)

Behavior : Declaring a bucket here enables input access to objects the application does not own:

- **S3 URI and S3 HTTP URL access** — `s3://` URIs and S3 HTTP URLs (including presigned URLs) pointing at these buckets are accepted as input sources; an HTTP URL is automatically converted to an `s3://` URI so Bedrock can access the object directly.
- **Declared region** — The region mapped to each bucket is used to reach that bucket in its own region when reading the input object. It does not influence model or inference region selection.

Without this setting, only the application's own buckets (`AWS_S3_BUCKET` and `AWS_S3_REGIONAL_BUCKETS`) are recognized.

export AWS_S3_ACCEPTED_BUCKETS='{"my-data-bucket": "us-east-1", "my-eu-bucket": "eu-west-1"}'

{
  "Effect": "Allow",
  "Action": "s3:GetObject",
  "Resource": [
    "arn:aws:s3:::my-data-bucket/*",
    "arn:aws:s3:::my-eu-bucket/*"
  ]
}

S3 Bucket Lifecycle Configuration¶

Purpose : Configure automatic deletion of temporary files and abandoned multipart upload parts to minimize storage costs

Recommendation : Configure S3 lifecycle policies to automatically delete objects under the AWS_S3_TMP_PREFIX after 1 day, and abort incomplete multipart uploads under the AWS_S3_FILES_PREFIX after 1 day

stdapi.ai stores temporary files under the prefix configured by AWS_S3_TMP_PREFIX (default: tmp/). These include generated images, audio files, and transcription workflow files. Configure S3 lifecycle policies to automatically delete objects under this prefix after 1 day.

Additionally, multipart file uploads (OpenAI Uploads API) store parts under AWS_S3_FILES_PREFIX (default: files/). If a session is never completed or cancelled — for example when a client disconnects — the uploaded parts remain in S3 and accumulate costs. Add an AbortIncompleteMultipartUpload rule on the files prefix to clean these up automatically.

{
  "Rules": [
    {
      "Id": "DeleteTemporaryFiles",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "tmp/"
      },
      "Expiration": {
        "Days": 1
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 1
      }
    },
    {
      "Id": "AbortIncompleteMultipartUploads",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "files/"
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 1
      }
    }
  ]
}

Apply via AWS CLI:

# For primary S3 bucket (AWS_S3_BUCKET)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-stdapi-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

# For transcribe S3 bucket (AWS_TRANSCRIBE_S3_BUCKET, if different from AWS_S3_BUCKET)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-transcribe-temp-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

# For regional buckets (AWS_S3_REGIONAL_BUCKETS)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-stdapi-us-west-2-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

Bedrock Configuration¶

AWS_BEDROCK_REGIONS¶

Purpose : List of AWS regions where Bedrock models are available

Format : Comma-separated string

Default : Current AWS SDK region if not specified

Behavior : Models are discovered in the same order as the listed regions. The first region is the primary region where your server should be hosted on AWS for optimal performance. Your S3 bucket (AWS_S3_BUCKET) must also be in this region. If a model is unavailable in the primary region, subsequent regions are checked in order

export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,eu-west-1

AWS_BEDROCK_CROSS_REGION_INFERENCE¶

Purpose : Enable automatic cross-region routing when a model isn't available in the primary region

Type : Boolean

Default : true

export AWS_BEDROCK_CROSS_REGION_INFERENCE=true

AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL¶

Purpose : Allow global cross-region inference routing to any region worldwide

Type : Boolean

Default : true

export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

AWS_BEDROCK_REGION_ROUTING¶

Purpose : Automatic region routing strategy for distributing Bedrock requests across configured regions

Type : String

Default : ordered

Behavior : When multiple regions are configured in AWS_BEDROCK_REGIONS, this setting controls how requests are distributed across them. The router automatically handles quota/throttling errors and regional unavailability by temporarily avoiding affected regions

Requirement : Requires at least 2 regions in AWS_BEDROCK_REGIONS to take effect

Available strategies:

# Use ordered routing (default)
export AWS_BEDROCK_REGION_ROUTING=ordered

# Use lowest latency routing
export AWS_BEDROCK_REGION_ROUTING=lowest_latency

# Disable routing
export AWS_BEDROCK_REGION_ROUTING=disabled

AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS¶

Purpose : Duration to temporarily avoid a region after receiving a quota or throttling error

Type : Integer (seconds, must be > 0)

Default : 60

Behavior : This is the base backoff value. When a Bedrock API call fails due to quota limits (ThrottlingException, TooManyRequestsException, ServiceQuotaExceededException), the affected region is temporarily blocked. The actual delay doubles with each consecutive quota error on the same region (exponential backoff), up to a hard ceiling of 1 hour. The counter resets after a successful request. Subsequent requests are routed to other available regions during the backoff period.

# Default: 60 seconds (base value — actual delay doubles per consecutive error)
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=60

# Shorter base backoff for aggressive retry
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=30

# Longer base backoff for conservative approach
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=120

AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS¶

Purpose : Duration to temporarily avoid a region after receiving an unavailability error

Type : Integer (seconds, must be > 0)

Default : 30

Behavior : When a Bedrock API call fails due to service unavailability (ServiceUnavailableException, ModelNotReadyException), the affected region is temporarily blocked for this many seconds. These errors are typically shorter-lived than quota limits, so the default is shorter

# Default: 30 seconds
export AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS=30

# Longer backoff for stability
export AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS=60

AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS¶

Purpose : Hard ceiling in seconds on the exponential quota backoff for a single region

Type : Integer (seconds, must be > 0)

Default : 3600 (1 hour)

Behavior : Quota backoff grows exponentially with consecutive errors (base interval × 2^n). This setting caps how large that value can become, preventing a region from being blocked indefinitely. Reduce it to allow faster recovery; increase it to keep a misbehaving region sidelined for longer.

# Default: 1 hour ceiling
export AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS=3600

# More aggressive recovery
export AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS=600

AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR¶

Purpose : Multiplier applied to the max quota backoff to compute the stale-error reset threshold

Type : Integer (must be > 0)

Default : 2 (threshold = 2 × max quota backoff = 2 hours with defaults)

Behavior : If the most recent quota error on a region occurred more than max_quota_backoff × factor seconds ago, the consecutive-error counter is reset and the next error is treated as a fresh start rather than an escalation. A higher value keeps memory of past errors for longer before resetting the counter.

# Default: reset counter after 2× the max backoff window
export AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR=2

# Longer memory of past errors
export AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR=4

AWS_BEDROCK_MAX_RETRIES¶

Purpose : Maximum number of retries per Bedrock invocation, each retry escalating to the next available region

Type : Integer (must be 0 or greater; 0 disables retries)

Default : 9

Behavior : Controls the retry budget for each Bedrock API call. When region routing is enabled, every retry escalates to the next region in priority order and each candidate region is tried at most once, so the attempts are bounded by the smaller of AWS_BEDROCK_MAX_RETRIES + 1 and the number of candidate regions for the model — with 3 regions and the default 9 retries, a request makes at most 3 attempts. A region that just failed is still blocked by its own backoff, and retrying it would only extend that backoff instead of recovering the request. When routing is disabled, or the region is pinned by S3 inputs, the full budget is spent as SDK retries against that single region.

# Default: 9 retries (10 total attempts)
export AWS_BEDROCK_MAX_RETRIES=9

# Fail faster (e.g. low-latency interactive use cases)
export AWS_BEDROCK_MAX_RETRIES=3

# Deeper in-region retrying for single-region or S3-pinned requests
export AWS_BEDROCK_MAX_RETRIES=18

AWS_FAILOVER_MAX_RETRIES¶

Purpose : Maximum SDK retry attempts per candidate region for the multi-region failover services (Polly, Transcribe, Translate, Comprehend)

Type : Integer (must be 0 or greater)

Default : 2

Behavior : Only applied when a service has several candidate regions (no explicit region setting): each region attempt uses this reduced retry budget (2 retries = 3 attempts per region) before failing over, so failover across regions replaces deep in-region retrying. When a service is pinned to a single region, the standard retry budget from AWS_BEDROCK_MAX_RETRIES applies instead.

# Default: 2 retries (3 attempts) per candidate region
export AWS_FAILOVER_MAX_RETRIES=2

# Fail over after a single attempt per region
export AWS_FAILOVER_MAX_RETRIES=0

AWS_BEDROCK_MANTLE_ENABLED¶

Purpose : Expose models served by the Amazon Bedrock Mantle endpoint (OpenAI/Anthropic-compatible APIs) in addition to the classic Bedrock Converse models

Type : Boolean

Default : true

Behavior : Mantle-only models (e.g. OpenAI GPT, xAI Grok, Google Gemma 4) become available on the chat completions, responses, messages, and completions routes. Models available on both the classic bedrock-runtime endpoint and Mantle are served by bedrock-runtime unless listed in AWS_BEDROCK_MANTLE_PREFERRED_MODELS.

Authentication requires no static secrets: short-term bearer tokens are derived automatically (SigV4-presigned) from the same AWS credential chain the server already uses, and refreshed transparently.

When Bedrock Mantle is unreachable or the IAM role lacks `bedrock-mantle` permissions, Mantle models are simply not listed and a warning is logged at startup — no configuration change required.

export AWS_BEDROCK_MANTLE_ENABLED=false

Bedrock Mantle Models feature overview

AWS_BEDROCK_MANTLE_REGIONS¶

Purpose : List of AWS regions used for Amazon Bedrock Mantle, in failover priority order

Type : Comma-separated string of AWS region identifiers

Default : The regions of AWS_BEDROCK_REGIONS that offer Bedrock Mantle

Behavior : Model availability differs per region; the served model catalog is the union of all listed regions. Region failover, quota backoff, and health tracking work exactly like classic Bedrock region routing.

export AWS_BEDROCK_MANTLE_REGIONS=us-east-1,eu-west-1

AWS_BEDROCK_MANTLE_ENDPOINT_URL¶

Purpose : Override the Amazon Bedrock Mantle endpoint URL template

Type : String — URL template with a {region} placeholder

Default : None (https://bedrock-mantle.{region}.api.aws)

Behavior : The {region} placeholder is substituted with the target region.

export AWS_BEDROCK_MANTLE_ENDPOINT_URL='https://bedrock-mantle.{region}.api.aws'

AWS_BEDROCK_MANTLE_PREFERRED_MODELS¶

Purpose : Model IDs (or ID prefixes) served by Amazon Bedrock Mantle even when also available on the classic bedrock-runtime endpoint

Type : Comma-separated string of model IDs or ID prefixes

Default : [] (empty — dual-homed models are served by bedrock-runtime)

Behavior : Useful to leverage Mantle's independent throughput quotas or native response storage for selected models. Mantle quotas (per-model, per-region tokens-per-minute) are independent from bedrock-runtime quotas.

export AWS_BEDROCK_MANTLE_PREFERRED_MODELS='anthropic.claude-haiku-4-5,openai.gpt-oss'

AWS_BEDROCK_MANTLE_SERVICE_HEADER¶

Purpose : Honor the x-stdapi-service: bedrock-mantle request header to route a model available on both endpoints through Bedrock Mantle for that request instead of the default bedrock-runtime serving

Type : Boolean

Default : false

export AWS_BEDROCK_MANTLE_SERVICE_HEADER=true

AWS_BEDROCK_MANTLE_PROJECT¶

Purpose : Default Amazon Bedrock Project/Workspace ID attributed to Bedrock Mantle inference requests for cost tracking and observability

Type : String — a bare project ID (e.g. proj_abc123 or default), not an ARN

Default : None (requests fall to the account's default project)

Behavior : Bedrock Projects (OpenAI-compatible APIs) and Workspaces (Anthropic Messages API) are the same underlying resource; the value is sent as the OpenAI-Project header on the Chat Completions and Responses APIs, and as the anthropic-workspace header on the Anthropic Messages API. When unset, requests fall to the account's default project — no failure.

export AWS_BEDROCK_MANTLE_PROJECT=proj_abc123

AWS_BEDROCK_ALLOW_MANTLE_PROJECT_OVERRIDE¶

Purpose : Allow a request to override the configured Mantle project via the OpenAI-Project / anthropic-workspace header

Type : Boolean

Default : false

Behavior : When true, a request may set its own project through the OpenAI-Project (Chat Completions, Responses) or anthropic-workspace (Anthropic Messages) header. When false and AWS_BEDROCK_MANTLE_PROJECT is configured, the request header is ignored and the server default applies. When no default project is configured, the request header is always honored regardless of this flag. A malformed request-supplied project ID returns 400.

export AWS_BEDROCK_ALLOW_MANTLE_PROJECT_OVERRIDE=true

AWS_BEDROCK_EXTERNAL_WEB_ACCESS¶

Purpose : Let the built-in web search tool reach the public web

Type : Boolean

Default : false

Behavior : Controls whether the built-in web search tool may reach the external web. Searches are answered from the Amazon Bedrock web index and cache either way, and answers are current and carry source citations. AWS documents that retrieval is served entirely from that index and cache today, so no request data leaves the AWS boundary even when this is enabled, and that a future release may allow live external retrieval — at which point request data may leave it. Enabling it is therefore a decision taken in advance about behaviour that can change. It also requires the bedrock-websearch:ExternalWebAccess IAM permission on the credentials this server uses; each action is authorized only when a model actually attempts it, and a denied call does not fail the request.

export AWS_BEDROCK_EXTERNAL_WEB_ACCESS=true

AWS_BEDROCK_ALLOW_EXTERNAL_WEB_ACCESS_OVERRIDE¶

Purpose : Allow a request to override AWS_BEDROCK_EXTERNAL_WEB_ACCESS with the external_web_access extra model parameter

Type : Boolean

Default : false

Behavior : When true, a request that sends external_web_access as an extra model parameter decides for that request, on the models whose web search takes a web access choice per request — the OpenAI GPT-5.x family. When false, a request that sets it to anything other than the configured value is rejected with 400 rather than being silently overridden; a request that omits it always gets the configured value. A request asking for a value a model cannot be given is rejected with 400 as well, rather than accepted and quietly ignored. On the models that do take it, a request naming no web search tool is accepted: nothing is searched, so the value has no search to apply to.

export AWS_BEDROCK_ALLOW_EXTERNAL_WEB_ACCESS_OVERRIDE=true

AWS_BEDROCK_MODEL_REGION_RESTRICT¶

Purpose : Restrict a model to specific region(s) only, useful when a model provides important features only in certain regions

Type : JSON object (keys: Bedrock model IDs or prefixes, values: ordered lists of allowed regions)

Default : {} (empty — no model-specific region restriction)

Behavior : When set, the model is made available only in the listed regions (intersected with the regions where it is actually available), and the list order defines the routing priority when the default ordered routing strategy is used. No fallback to other regions occurs. Keys can be exact model IDs or prefixes that match the beginning of a model ID

# Restrict Nova Pro to us-east-1 for grounding support
export AWS_BEDROCK_MODEL_REGION_RESTRICT='{"amazon.nova-pro-v1:0": ["us-east-1"]}'

AWS_BEDROCK_LEGACY¶

Purpose : Allow usage of legacy/deprecated Bedrock models

Type : Boolean

Default : false

export AWS_BEDROCK_LEGACY=true

AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK¶

Purpose : Transparently reroute requests using a deprecated model ID to its recommended replacement

Type : Boolean

Default : true

Behavior : When true, any request that specifies a deprecated model ID (as listed in the server's deprecation registry) is silently retried with the recommended replacement model. The replacement is fully re-evaluated — alias resolution, modality checks, and region routing all apply to the new model ID. When false, deprecated model IDs return a 404 error with a message indicating the replacement, forcing clients to migrate explicitly.

# Transparent fallback (default) — clients using old model IDs keep working
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=true

# Strict mode — deprecated model IDs return 404, clients must update their code
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=false

AWS_BEDROCK_DEPRECATED_MODELS¶

Purpose : Extend or override the built-in deprecated model registry with custom mappings

Type : JSON object — dict[str, str]

Default : {}

Behavior : Merged with the built-in registry at startup. User-provided entries take precedence over built-in ones — this means it can be used both to add new deprecated model mappings and to override the fallback target of an already-defined deprecated model. Effective only when AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK is true.

Reference : Amazon Bedrock model lifecycle

# Add a custom deprecated model and override an existing built-in mapping
export AWS_BEDROCK_DEPRECATED_MODELS='{"my-old-model-v1": "my-new-model-v2", "amazon.titan-text-lite-v1": "amazon.nova-lite-v1:0"}'

AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE¶

Purpose : Control automatic subscription to new models in AWS Marketplace

Type : Boolean

Default : true

Behavior : When true, the server automatically subscribes to new models discovered in the AWS Marketplace, making them immediately available through the API. When false, only models with existing marketplace subscriptions are visible and accessible

IAM Permissions Required : aws-marketplace:Subscribe, aws-marketplace:ViewSubscriptions — see Marketplace Auto-Subscribe IAM

# Allow automatic subscription (default)
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=true

# Restrict to pre-subscribed models only
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=false

AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass cross-region inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use cross-region inference profile ARNs instead of model IDs in the model parameter. Cross-region inference profiles enable routing to multiple regions for better availability

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable cross-region inference profile ARN support
export AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-sonnet-5

AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass application inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use application inference profile ARNs instead of model IDs in the model parameter. Application inference profiles are custom routing configurations for specific use cases

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable application inference profile ARN support
export AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123xyz

AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN¶

Purpose : Allow users to pass prompt router ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use prompt router ARNs instead of model IDs in the model parameter. Prompt routers enable dynamic model selection based on prompt characteristics

IAM Permissions Required : bedrock:GetPromptRouter (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable prompt router ARN support
export AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN=true

arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router

AWS_BEDROCK_ALLOW_PROMPT_ARN¶

Purpose : Allow users to reference an Amazon Bedrock Prompt Management prompt ARN in the OpenAI Responses API prompt parameter

Type : Boolean

Default : false

Behavior : When enabled, prompt.id accepts a prompt ARN (with an optional prompt.version) and prompt.variables fill in the template. Amazon Bedrock renders the stored prompt, and the model it is bound to serves the request. When disabled, any prompt parameter is rejected with a 400 error

IAM Permissions Required : bedrock:GetPrompt (resolve the prompt's model) and bedrock:RenderPrompt (invoke it)

# Disabled (default) - the Responses API `prompt` parameter returns 400
# No environment variable needed

# Enable Prompt Management prompt ARN support
export AWS_BEDROCK_ALLOW_PROMPT_ARN=true

arn:aws:bedrock:us-east-1:123456789012:prompt/ABCDE12345:1

AWS_BEDROCK_MODEL_ARN_MAPPING¶

Purpose : Map standard model IDs to custom inference profile or prompt router ARNs for server-controlled routing

Format : JSON object with model IDs as keys and ARNs as values

Default : {} (empty, no mappings)

Behavior : When configured, the mapped ARN is used instead of the default cross-region inference profile when clients request the model by its standard ID. This provides centralized control over model routing without requiring client changes

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-custom-profile",
  "anthropic.claude-haiku-4-5-20251001-v1:0": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/cost-optimizer"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/production-profile"
}'

# Production: Use cost-optimized prompt router
export AWS_BEDROCK_MODEL_ARN_MAPPING='{"anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/prod-router"}'

# Development: Use standard cross-region profile
export AWS_BEDROCK_MODEL_ARN_MAPPING='{}'

Other AWS Services¶

AWS_POLLY_REGION¶

Purpose : Region for Amazon Polly text-to-speech service

Default : All regions in AWS_BEDROCK_REGIONS, with per-engine regional discovery and automatic failover

Behavior : When unset, voice availability is discovered per engine in every AWS_BEDROCK_REGIONS entry at startup: an engine (Standard, Neural, Long-form, Generative) is exposed as a model when at least one candidate region offers it, and each synthesis call routes to the regions offering the requested engine and voice, failing over on region-level errors. Setting an explicit region pins Polly to that single region — engines it does not offer are then disabled.

export AWS_POLLY_REGION=us-east-1

AWS_COMPREHEND_REGION¶

Purpose : Region for the Amazon Comprehend services (language detection and toxicity moderation)

Default : All regions in AWS_BEDROCK_REGIONS, tried in order with automatic failover

Behavior : When unset, Comprehend calls try each AWS_BEDROCK_REGIONS entry in order and fail over to the next region on region-level errors (throttling, service unavailability, network issues, or a region that does not offer Comprehend or the requested operation). Setting an explicit region pins Comprehend to that single region with no failover.

export AWS_COMPREHEND_REGION=us-east-1

AWS_TRANSCRIBE_REGION¶

Purpose : Region for Amazon Transcribe speech-to-text service

Default : All regions in AWS_BEDROCK_REGIONS that have a co-located S3 bucket, tried in order with automatic failover

Behavior : Transcription jobs need an S3 bucket in the job's region. When unset, every AWS_BEDROCK_REGIONS entry with a usable bucket is a candidate — the primary region is served by AWS_TRANSCRIBE_S3_BUCKET (or AWS_S3_BUCKET), the others by their AWS_S3_REGIONAL_BUCKETS entry. On a region-level error while starting a job, the audio is server-side copied to the next candidate's bucket and the job restarts there. Setting an explicit region pins Transcribe to that single region with no failover.

export AWS_TRANSCRIBE_REGION=us-east-1

AWS_TRANSLATE_REGION¶

Purpose : Region for Amazon Translate text translation service

Default : All regions in AWS_BEDROCK_REGIONS, tried in order with automatic failover

Behavior : When unset, translation calls try each AWS_BEDROCK_REGIONS entry in order and fail over to the next region on region-level errors (throttling, service unavailability, network issues, or a region that does not offer Translate). Setting an explicit region pins Translate to that single region with no failover.

export AWS_TRANSLATE_REGION=us-east-1

Compliance and Latency Optimization¶

Strategic region configuration is critical for both regulatory compliance and performance optimization. This section provides best practice configurations for common scenarios.

GDPR and Data Residency Compliance¶

For applications serving European users, data residency regulations like GDPR may require that data processing occurs within specific geographic boundaries.

# Use only European regions
export AWS_S3_BUCKET=my-stdapi-eu-bucket
export AWS_BEDROCK_REGIONS=eu-west-1,eu-west-3,eu-central-1

# Disable global cross-region inference to prevent data routing outside Europe
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

# Keep cross-region inference enabled for failover within EU regions
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true

Latency Optimization¶

For applications prioritizing low latency and high performance, configure regions closest to your users and application infrastructure.

North America:

# Primary region for lowest latency, with fallbacks
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket
export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,us-east-2

# Enable all cross-region inference for maximum model availability
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Asia-Pacific:

# Use Asia-Pacific regions for lowest latency to APAC users
export AWS_S3_BUCKET=my-stdapi-ap-southeast-1-bucket
export AWS_BEDROCK_REGIONS=ap-southeast-1,ap-northeast-1,us-west-2

# Enable global inference for fallback to US regions if needed
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Global Multi-Region:

# Balanced configuration with worldwide coverage
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket
export AWS_BEDROCK_REGIONS=us-east-1,eu-west-1,ap-southeast-1,us-west-2

# Enable global inference for best availability
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Hybrid Approach: Compliance with Performance¶

Balance compliance requirements with performance needs:

# EU primary with US fallback (for model availability)
export AWS_S3_BUCKET=my-stdapi-eu-bucket
export AWS_BEDROCK_REGIONS=eu-west-1,eu-central-1,us-east-1

# Allow cross-region but restrict to specific regions only
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

Configuration Order¶

When deploying stdapi.ai, configure settings in this recommended order:

1. IAM Permissions - Set up AWS access first
2. AWS Services and Regions - Configure S3 buckets and Bedrock regions
3. Authentication - Secure your API with authentication
4. Optional Features - Add observability, guardrails, and other features as needed

IAM Permissions¶

The full IAM reference — required Amazon Bedrock permissions, per-feature policy statements, complete policy examples, and AWS tag policy requirements — has moved to the dedicated IAM Permissions page.

Authentication¶

stdapi.ai supports three sources for API key authentication, plus Amazon Cognito user pool tokens.

Method 1: Direct API Key¶

Provide the API key directly via environment variable. Intended for local development and testing; it takes precedence over both AWS-backed sources.

API_KEY¶

Purpose : Static API key value

Security Warning : Avoid hardcoding in configuration files; use environment variables only

Client Usage : Clients must include this key in the Authorization: Bearer <key> header or X-API-Key header

export API_KEY=sk-1234567890abcdef...

Method 2: SSM Parameter Store (Recommended)¶

Recommended - Use AWS Systems Manager Parameter Store for secure key storage with encryption, access control, and auditing. This method should be used only with already existing parameters.

API_KEY_SSM_PARAMETER¶

Purpose : Name of the SSM parameter containing the API key. The parameter is retrieved from the current region detected by the running container, or defaults to the first region in AWS_BEDROCK_REGIONS.

Recommendation : Use SecureString type for encryption at rest

IAM Permissions Required : ssm:GetParameter, kms:Decrypt (if encrypted)

export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

Method 3: Secrets Manager¶

Use AWS Secrets Manager for secure key storage with automatic rotation support. This method should be used only with already existing secrets.

API_KEY_SECRETSMANAGER_SECRET¶

Purpose : Name of the Secrets Manager secret containing the API key. The secret is retrieved from the current region detected by the running container, or defaults to the first region in AWS_BEDROCK_REGIONS.

Format : Can be a plain string or JSON object

IAM Permissions Required : secretsmanager:GetSecretValue

API_KEY_SECRETSMANAGER_KEY¶

Purpose : JSON key name within the secret (if the secret is a JSON object)

Default : api_key

Plain String Secret:

export API_KEY_SECRETSMANAGER_SECRET=stdapi-api-key

JSON Secret:

export API_KEY_SECRETSMANAGER_SECRET=stdapi-credentials
export API_KEY_SECRETSMANAGER_KEY=api_key

Example JSON secret structure:

{
  "api_key": "sk-1234567890abcdef...",
  "other_config": "value"
}

Method 4: Amazon Cognito User Pool Tokens¶

Accept the bearer tokens issued by an Amazon Cognito user pool instead of, or alongside, the API key. Each caller gets its own short-lived credential, and the verified caller is the identity per-user cost attribution bills against; withdrawing a caller's access takes effect when their current token expires. Clients send the token in the Authorization: Bearer <token> or X-API-Key header, like an API key. What is validated on every request is described in Authentication & Security.

export AWS_COGNITO_USER_POOL_ID=eu-west-3_a1b2c3d4e
export AWS_COGNITO_CLIENT_IDS=1example23456789abcdefghij

AUTHENTICATION_MODE¶

Purpose : Which client authentication methods the deployment accepts

Default : any — every method that is configured

Values : - any: the API key and user pool tokens, whichever is configured - api_key: the API key only; startup fails if a user pool is also configured - cognito: user pool tokens only; startup fails if an API key source is also configured

export AUTHENTICATION_MODE=cognito

AWS_COGNITO_USER_POOL_ID¶

Purpose : Identifier of the user pool whose tokens authenticate clients. Setting it enables the method; the pool's AWS Region is read from the identifier itself, and the public signing keys are loaded from that Region at startup.

Default : None — user pool tokens are not accepted

Requirement : AWS_COGNITO_CLIENT_IDS must be set too

IAM Permissions Required : None — the signing keys are public

export AWS_COGNITO_USER_POOL_ID=eu-west-3_a1b2c3d4e

AWS_COGNITO_CLIENT_IDS¶

Purpose : Comma-separated app client IDs whose tokens are accepted. A token issued to any other app client of the pool is rejected.

Default : Empty — startup fails when a user pool is configured without it

Requirement : Required whenever AWS_COGNITO_USER_POOL_ID is set

export AWS_COGNITO_CLIENT_IDS=1example23456789abcdefghij,2example3456789abcdefghijk

AWS_COGNITO_REQUIRED_SCOPES¶

Purpose : Comma-separated OAuth 2.0 scopes a token must all carry to be accepted

Default : None — any scope set is accepted

Requirement : Custom scopes exist only on tokens issued by the pool's OAuth 2.0 token endpoint, which needs a resource server and a pool domain. Tokens obtained by signing in with a username and password carry only aws.cognito.signin.user.admin and are rejected when a custom scope is required.

export AWS_COGNITO_REQUIRED_SCOPES=stdapi/invoke

AWS_COGNITO_ACCEPT_ID_TOKEN¶

Purpose : Also accept identity tokens, not only access tokens

Default : false

Effect : Identity tokens describe the signed-in user rather than granting API access, and carry no scopes. Enable only for clients that cannot obtain an access token.

export AWS_COGNITO_ACCEPT_ID_TOKEN=true

AWS_COGNITO_ISSUER_TYPE¶

Purpose : The pool's issuer configuration, which decides the issuer URL its tokens carry

Default : original

Values : - original: https://cognito-idp.<region>.amazonaws.com/<pool-id> - updated: https://issuer-cognito-idp.<region>.amazonaws.com/<pool-id>, available on the Essentials and Plus pool tiers

Requirement : Must match the pool's own setting; tokens whose issuer differs are rejected

export AWS_COGNITO_ISSUER_TYPE=updated

Authentication Discovery for Agents¶

Publishes, at /.well-known/oauth-protected-resource, where clients obtain a token, and points every 401 Unauthorized at that document. An AI agent — or any MCP client — can then authenticate against this deployment without having been configured for it first. See Authentication Discovery for Agents for the full flow.

Nothing is published until OAUTH_RESOURCE_IDENTIFIER is set. With an Amazon Cognito user pool configured, that variable is the only one to set: the pool already names the issuer and the scopes, and both are published from it. The document is public and unauthenticated, since a client reads it before it has any credential.

OAUTH_RESOURCE_IDENTIFIER¶

Purpose : Public URL clients use to reach this deployment, published as the identity of the protected resource

Default : None — no discovery document is published, and 401 responses only state that a bearer token is expected

Requirement : Must be the exact origin clients dial — scheme and host, an explicit port only when it is not the default one for the scheme, and no path, query or fragment. Clients compare it character by character against the URL they used, so https://api.example.com and https://api.example.com:443 are not interchangeable. Requires OAUTH_AUTHORIZATION_SERVERS, unless a user pool supplies the issuer.

export OAUTH_RESOURCE_IDENTIFIER=https://api.example.com

OAUTH_AUTHORIZATION_SERVERS¶

Purpose : Issuer URLs of the OAuth 2.0 authorization servers that issue tokens for this deployment, comma-separated

Default : The issuer of the AWS_COGNITO_USER_POOL_ID pool, when one is configured — otherwise none

Effect : A client reads each issuer's own metadata to find where to sign in, so this deployment never describes the sign-in flow itself. A load balancer or API gateway authenticating in front of stdapi.ai publishes the issuer of whichever provider it uses.

With a user pool configured, leave this unset: the pool issues the tokens the deployment accepts, so its own issuer is published — `https://cognito-idp.<region>.amazonaws.com/<pool-id>`, or `https://issuer-cognito-idp.<region>.amazonaws.com/<pool-id>` when [`AWS_COGNITO_ISSUER_TYPE`](#aws-cognito-issuer-type) is `updated`. `<region>` and `<pool-id>` come from the pool ID itself, and the host follows the pool Region's AWS partition (`amazonaws.com.cn` in China, `amazonaws.eu` in the European Sovereign Cloud). Set the variable only to publish further issuers.

Requirement : Each entry is an https URL with no query or fragment. Required when OAUTH_RESOURCE_IDENTIFIER is set and no user pool is configured. When one is, the list must include the pool's own issuer — a client sent anywhere else obtains a token every request refuses, so startup fails instead.

export OAUTH_AUTHORIZATION_SERVERS=https://cognito-idp.eu-west-3.amazonaws.com/eu-west-3_a1b2c3d4e

OAUTH_SCOPES_SUPPORTED¶

Purpose : Scopes a token needs to call this API, comma-separated

Default : AWS_COGNITO_REQUIRED_SCOPES — with neither set, no scope is advertised and a client asks for whatever its own configuration names

Effect : Advertised both in the discovery document and in the 401 challenge, so a client asks its authorization server for the right scopes on its first attempt. The scopes a token must carry to be accepted are exactly the scopes to ask for, so they are published unless this variable names others.

export OAUTH_SCOPES_SUPPORTED=stdapi/invoke

API Compatibility¶

Configure the base URL paths for OpenAI and Anthropic-compatible API routes.

OPENAI_ROUTES_PREFIX¶

Purpose : Base path prefix for OpenAI-compatible API routes

Default : `` (empty, routes mounted at root)

Requirement : Empty, or a path starting with / with no trailing slash, using only alphanumeric characters and . _ ~ - per segment; must differ from ANTHROPIC_ROUTES_PREFIX and COHERE_ROUTES_PREFIX

Effect : All OpenAI-compatible endpoints will be mounted under this prefix

export OPENAI_ROUTES_PREFIX=/api

ANTHROPIC_ROUTES_PREFIX¶

Purpose : Base path prefix for Anthropic-compatible API routes

Default : /anthropic

Requirement : A path starting with / with no trailing slash, using only alphanumeric characters and . _ ~ - per segment; must differ from OPENAI_ROUTES_PREFIX and COHERE_ROUTES_PREFIX

Effect : All Anthropic-compatible endpoints will be mounted under this prefix

export ANTHROPIC_ROUTES_PREFIX=/anthropic

export ANTHROPIC_ROUTES_PREFIX=/api/anthropic

COHERE_ROUTES_PREFIX¶

Purpose : Base path prefix for Cohere-compatible API routes

Default : /cohere

Requirement : A path starting with / with no trailing slash, using only alphanumeric characters and . _ ~ - per segment; must differ from OPENAI_ROUTES_PREFIX and ANTHROPIC_ROUTES_PREFIX

Effect : All Cohere-compatible endpoints will be mounted under this prefix

export COHERE_ROUTES_PREFIX=/cohere

CORS Configuration¶

Configure Cross-Origin Resource Sharing (CORS) to control which web origins can access your API from browsers.

CORS_ALLOW_ORIGINS¶

Purpose : List of origins allowed to make cross-origin requests

Format : JSON array of origin URLs

Default : None (CORS not enabled)

Best Practice : Only enable if your API is accessed from web browsers; specify exact origins in production

# Not configured (default) - CORS middleware not enabled
# Browser cross-origin requests will be blocked
# No environment variable needed

# Development: Allow all origins
export CORS_ALLOW_ORIGINS='["*"]'

# Production: Specific origins only
export CORS_ALLOW_ORIGINS='["https://myapp.com", "https://app.example.com"]'

# Multiple environments
export CORS_ALLOW_ORIGINS='["https://app.example.com", "https://staging.example.com"]'

Trusted Host Configuration¶

Configure Host header validation to protect against Host header injection attacks.

TRUSTED_HOSTS¶

Purpose : List of trusted Host header values for validation

Format : JSON array of hostnames (supports wildcards)

Default : None (no Host header validation)

Best Practice : Use AWS ALB host-based routing rules instead when possible for better performance and management

# Production: Specific hosts only
export TRUSTED_HOSTS='["api.example.com", "www.example.com"]'

export TRUSTED_HOSTS='["*.example.com", "*.myapp.com", "api.production.com"]'

export TRUSTED_HOSTS='["api.example.com", "localhost", "127.0.0.1"]'

aws elbv2 create-rule \
  --listener-arn arn:aws:elasticloadbalancing:... \
  --priority 1 \
  --conditions Field=host-header,Values=api.example.com \
  --actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:...

Proxy Headers Configuration¶

Configure X-Forwarded-* header processing when running behind reverse proxies or load balancers.

ENABLE_PROXY_HEADERS¶

Purpose : Enable trusting X-Forwarded-* headers from reverse proxies

Type : Boolean

Default : false (disabled)

Best Practice : Only enable when running behind a trusted reverse proxy

# Disabled (default) - do not trust X-Forwarded-* headers
# No environment variable needed

# Enable when behind reverse proxy
export ENABLE_PROXY_HEADERS=true

# Do NOT enable proxy headers
# ENABLE_PROXY_HEADERS should remain false (default)

export ENABLE_PROXY_HEADERS=true

export ENABLE_PROXY_HEADERS=true

PROXY_TRUSTED_HOSTS¶

Purpose : Restrict which peer IPs may set trusted X-Forwarded-* headers when ENABLE_PROXY_HEADERS is enabled

Type : JSON array of IPs/CIDRs, or *

Default : * (trust every peer — backward compatible)

Best Practice : Restrict to your reverse proxy's IP range so direct clients cannot spoof X-Forwarded-For

# Trust forwarded headers only from the VPC / proxy range
export ENABLE_PROXY_HEADERS=true
export PROXY_TRUSTED_HOSTS='["10.0.0.0/8"]'

export PROXY_TRUSTED_HOSTS='["10.0.0.0/16", "::ffff:10.0.0.0/112"]'

TLS / SSL Configuration¶

Configure end-to-end TLS encryption within the container. These are native Granian environment variables and are available with the provided container images.

GRANIAN_SSL_CERTIFICATE¶

Purpose : Path to the SSL certificate file

Type : File path

GRANIAN_SSL_KEYFILE¶

Purpose : Path to the SSL private key file (PKCS#8 format only)

Type : File path

GRANIAN_SSL_KEYFILE_PASSWORD¶

Purpose : Password for the private key file

Type : String

GRANIAN_SSL_PROTOCOL_MIN¶

Purpose : Minimum supported TLS version (tls1.2 or tls1.3)

Type : Enum

Default : tls1.3

GRANIAN_SSL_CA¶

Purpose : Path to the CA certificate bundle used to verify client certificates (mTLS)

Type : File path

GRANIAN_SSL_CLIENT_VERIFY¶

Purpose : Enable client certificate verification (mTLS)

Type : Boolean

Default : false (disabled)

GZip Compression¶

Configure automatic GZip compression for HTTP responses to reduce bandwidth usage and improve response times.

ENABLE_GZIP¶

Purpose : Enable GZip compression for HTTP responses

Type : Boolean

Default : false (disabled)

Best Practice : Use AWS ALB or CloudFront compression instead when available for better performance

# Disabled (default) - no response compression
# No environment variable needed

# Enable GZip compression (responses larger than 1 KiB will be compressed)
export ENABLE_GZIP=true

MCP (Model Context Protocol)¶

When enabled, stdapi.ai exposes its API endpoints as MCP tools, allowing AI clients and agents to call them directly using the Model Context Protocol. The full list of available tool names is documented in API Overview → MCP Tools.

Both transport types can be enabled independently or simultaneously.

ENABLE_MCP_STREAMABLE_HTTP¶

Purpose : Enable the MCP server using Streamable HTTP transport — the recommended method

Type : Boolean

Default : false

Behavior : Exposes an MCP-compatible endpoint at /mcp. AI clients connect using standard HTTP requests following the MCP Streamable HTTP specification.

# Disabled (default)
# No environment variable needed

# Enable MCP Streamable HTTP transport
export ENABLE_MCP_STREAMABLE_HTTP=true

MCP_STATELESS_HTTP¶

Purpose : Serve the Streamable HTTP transport without server-side sessions

Type : Boolean

Default : false

Behavior : Each request to /mcp is handled by a fresh transport that keeps no state. Clients may call tools/list and tools/call without an initialize handshake, an Mcp-Session-Id the server never issued is accepted rather than rejected, and any replica may serve any request.

Requires : ENABLE_MCP_STREAMABLE_HTTP=true. Ignored otherwise.

# Sessions enabled (default)
# No environment variable needed

# Stateless transport
export ENABLE_MCP_STREAMABLE_HTTP=true
export MCP_STATELESS_HTTP=true

ENABLE_MCP_SSE¶

Purpose : Enable the MCP server using Server-Sent Events (SSE) transport

Type : Boolean

Default : false

Behavior : Exposes MCP endpoints at /sse for AI clients that require the SSE transport protocol.

# Disabled (default)
# No environment variable needed

# Enable MCP SSE transport
export ENABLE_MCP_SSE=true

export ENABLE_MCP_STREAMABLE_HTTP=true
export ENABLE_MCP_SSE=true

MCP_INCLUDE_TOOLS¶

Purpose : Expose only a specific subset of MCP tools; all others are hidden

Format : Comma-separated list of tool names (duplicates are automatically removed)

Default : None (all tools exposed)

# All tools exposed by default
# No environment variable needed

# Expose only specific tools
export MCP_INCLUDE_TOOLS="openai_chat_completion,openai_embedding,search_models"

# When both MCP_INCLUDE_TOOLS and MCP_EXCLUDE_TOOLS are specified,
# tools in MCP_EXCLUDE_TOOLS are removed from MCP_INCLUDE_TOOLS:
export MCP_INCLUDE_TOOLS="openai_chat_completion,openai_embedding,search_models"
export MCP_EXCLUDE_TOOLS="openai_files_delete,anthropic_files_delete"
# Result: only openai_chat_completion, openai_embedding, search_models are exposed

See API Overview → MCP Tools for the full list of available tool names.

MCP_EXCLUDE_TOOLS¶

Purpose : Hide specific MCP tools from clients; all others remain exposed

Format : Comma-separated list of tool names (duplicates are automatically removed)

Default : None (no tools excluded)

# No tools excluded by default
# No environment variable needed

# Exclude destructive tools
export MCP_EXCLUDE_TOOLS="openai_files_delete,anthropic_files_delete"

See API Overview → MCP Tools for the full list of available tool names.

Tool Selection Best Practices¶

stdapi.ai exposes a fixed set of tools derived from its API surface — you can include or exclude them by name, but cannot modify or rename them. See API Overview → MCP Tools for the full catalog.

Start from the minimum, not the maximum

By default all tools are exposed. It is safer and more effective to begin with a narrow MCP_INCLUDE_TOOLS list covering only what the workflow needs, then expand it deliberately. LLMs perform better with fewer choices, and many AI providers cap the number of active tools per session.

Always include search_models for agent model discovery

search_models is the recommended tool for agents to discover available model IDs — it supports capability-based filtering (by modality, route, region, streaming support) and returns richer metadata than openai_model_list or anthropic_model_list. Include it in every agent configuration so the agent can resolve the right model dynamically rather than relying on hardcoded IDs:

export MCP_INCLUDE_TOOLS="openai_chat_completion,search_models,openai_embedding"

Always exclude file deletion tools unless required

Uploaded files are the only durable, stateful data managed by stdapi.ai — deletion is permanent and cannot be undone. Unless your workflow explicitly needs to delete files, always suppress these tools:

export MCP_EXCLUDE_TOOLS="openai_files_delete,anthropic_files_delete"

Exclude high-cost tools unless the workflow requires them

Image generation (openai_image_generation, openai_image_edit, openai_image_variation) and speech synthesis (openai_audio_speech) incur a per-call cost that accumulates quickly if an agent invokes them speculatively. Only include them when the use case calls for it and the agent's decision to generate images or audio is intentional.

Use MCP_INCLUDE_TOOLS for the tightest control

For predictable, well-defined workflows, listing tools explicitly with MCP_INCLUDE_TOOLS is more reliable than maintaining an exclusion list. For example, a workflow limited to text generation and model discovery needs only:

export MCP_INCLUDE_TOOLS="openai_chat_completion,search_models"

SSRF Protection¶

Configure Server-Side Request Forgery (SSRF) protection to prevent unauthorized access to internal networks.

SSRF_PROTECTION_BLOCK_PRIVATE_NETWORKS¶

Purpose : Enable SSRF protection by blocking requests to private/local networks

Type : Boolean

Default : true (enabled for security)

Best Practice : Keep enabled in production to protect against SSRF attacks

# Enabled (default) - block private networks
# No environment variable needed

# Disable only in controlled environments that need local network access
export SSRF_PROTECTION_BLOCK_PRIVATE_NETWORKS=false

Request Limits¶

Bound per-request resource usage to protect the server when the API is exposed to untrusted clients.

MAX_INPUT_FILE_SIZE¶

Purpose : Cap the size of an inline input file loaded into memory to protect against memory-exhaustion (DoS)

Type : Integer (bytes)

Default : 0 (disabled — no limit)

Best Practice : Set a limit aligned with your largest expected inline input (e.g. 26214400 for 25 MiB) when the API is exposed to untrusted clients

# Disabled (default) - no size limit
# No environment variable needed

# Reject inline inputs larger than 25 MiB
export MAX_INPUT_FILE_SIZE=26214400

MAX_CONCURRENT_INPUT_DOWNLOADS¶

Purpose : Bound the number of input files fetched or resolved concurrently within a single request

Type : Integer (> 0)

Default : 8

Best Practice : Keep a modest value so a single request with many remote inputs cannot exhaust sockets/memory or amplify outbound requests against a target

# Allow up to 4 concurrent input downloads per request
export MAX_CONCURRENT_INPUT_DOWNLOADS=4

Observability (OpenTelemetry)¶

Configure distributed tracing for debugging and performance monitoring. stdapi.ai integrates with AWS X-Ray, Jaeger, DataDog, and other OTLP-compatible systems.

OTEL_ENABLED¶

Purpose : Enable or disable OpenTelemetry tracing

Type : Boolean

Default : false

export OTEL_ENABLED=true

OTEL_SERVICE_NAME¶

Purpose : Service identifier in trace visualizations

Default : stdapi.ai

Best Practice : Use descriptive names with environment information

export OTEL_SERVICE_NAME=stdapi-production-us-east-1

OTEL_EXPORTER_ENDPOINT¶

Purpose : OTLP HTTP endpoint URL for sending traces

Default : http://127.0.0.1:4318/v1/traces

Protocol : Must support OTLP HTTP format

AWS X-Ray (via ADOT):

export OTEL_EXPORTER_ENDPOINT=http://127.0.0.1:4318/v1/traces

Jaeger:

export OTEL_EXPORTER_ENDPOINT=http://jaeger:14268/api/traces

Cloud Provider OTLP:

# Use provider-specific OTLP endpoints
export OTEL_EXPORTER_ENDPOINT=https://your-provider-otlp-endpoint.com/v1/traces

OTEL_SAMPLE_RATE¶

Purpose : Percentage of requests to trace (controls cost vs. observability)

Type : Float (0.0 to 1.0)

Default : 1.0 (100%)

Development:

# Trace everything for debugging
export OTEL_SAMPLE_RATE=1.0

Production (Moderate Traffic):

# Sample 10% of requests
export OTEL_SAMPLE_RATE=0.1

Production (High Traffic):

# Sample 1% of requests
export OTEL_SAMPLE_RATE=0.01

API Documentation Routes¶

stdapi.ai provides automatic API documentation routes, which are disabled by default for security in production environments.

ENABLE_DOCS¶

Purpose : Enable interactive Swagger UI documentation at /docs

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_DOCS=true

ENABLE_REDOC¶

Purpose : Enable ReDoc documentation UI at /redoc

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_REDOC=true

ENABLE_OPENAPI_JSON¶

Purpose : Enable OpenAPI schema JSON endpoint at /openapi.json

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_OPENAPI_JSON=true

Development Configuration¶

Enable all documentation routes for local development:

export ENABLE_DOCS=true
export ENABLE_REDOC=true
# ENABLE_OPENAPI_JSON is automatically enabled when ENABLE_DOCS or ENABLE_REDOC is true

Or enable only Swagger UI:

export ENABLE_DOCS=true
# ENABLE_OPENAPI_JSON is automatically enabled

Or enable only ReDoc:

export ENABLE_REDOC=true
# ENABLE_OPENAPI_JSON is automatically enabled

Production Best Practice¶

# Keep all routes disabled in production (default)
# No environment variables needed - defaults to false

Validation and Logging¶

For comprehensive logging and monitoring information, see the Logging and Monitoring guide.

STRICT_INPUT_VALIDATION¶

Purpose : Reject API requests containing unknown/extra fields instead of ignoring them

Type : Boolean

Default : false

# Returns HTTP 400 for requests with unexpected fields
export STRICT_INPUT_VALIDATION=true

CHAT_COMPLETIONS_REASONING_FIELD¶

Purpose : Choose which field carries a reasoning model's thinking text on /v1/chat/completions

Type : String

Default : reasoning_content

Options : reasoning_content, reasoning, none

Behavior : The OpenAI Chat Completions API returns no thinking text of its own — it reports only a reasoning_tokens count — so the providers that do return it have settled on two different names. reasoning_content is the DeepSeek spelling, which most clients that read reasoning at all look for first. reasoning is the name used by OpenRouter and vLLM. none emits neither, keeping responses strictly OpenAI-shaped. : The setting applies to both the completed message and the streamed deltas, so a client never sees one name while streaming and another at the end. Callers can also suppress reasoning per request with include_reasoning: false or reasoning: {"exclude": true}, whatever this is set to.

# Default: the name most clients read
export CHAT_COMPLETIONS_REASONING_FIELD=reasoning_content

# For clients written against OpenRouter or vLLM
export CHAT_COMPLETIONS_REASONING_FIELD=reasoning

# Strict OpenAI shape: never return thinking text
export CHAT_COMPLETIONS_REASONING_FIELD=none

LOG_LEVEL¶

Purpose : Control the minimum severity of log events written to STDOUT

Default : info

Options : info, warning, error, critical, disabled

Behavior : Only log events at or above the configured level are output. Log levels are ordered by severity: info < warning < error < critical

# Default: Output all log events
export LOG_LEVEL=info

# Production: Suppress info logs, show only warnings and higher
export LOG_LEVEL=warning

# Critical only: Show only critical errors
export LOG_LEVEL=critical

# Disable logging: Suppress all log output (not recommended)
export LOG_LEVEL=disabled

LOG_REQUEST_PARAMS¶

Purpose : Include request and response parameters (JSON body, form, query) in logs for integration debugging

Type : Boolean

Default : false

# Enable for debugging (NOT recommended for production)
export LOG_REQUEST_PARAMS=true

LOG_CLIENT_IP¶

Purpose : Enable logging of client IP addresses for each request and add IP to OpenTelemetry spans

Type : Boolean

Default : false (disabled for privacy)

# Disabled (default) - no client IP logging
# No environment variable needed

# Enable client IP logging
export LOG_CLIENT_IP=true

# Enable proxy headers to get real client IPs
export ENABLE_PROXY_HEADERS=true
# Enable client IP logging
export LOG_CLIENT_IP=true

# Only enable client IP logging
export LOG_CLIENT_IP=true
# ENABLE_PROXY_HEADERS remains false (default)

TIMEZONE¶

Purpose : IANA timezone identifier used for request date and time

Type : String (IANA timezone identifier)

Default : UTC

# UTC (default)
export TIMEZONE=UTC

# North America
export TIMEZONE=America/New_York

# Europe
export TIMEZONE=Europe/London

CloudWatch Metrics and Cost Tracking¶

The behavior of these settings — EMF line structure, cost log format, pricing accuracy, regional price fallback, known limitations, and the price override format with examples — is documented in CloudWatch Metrics (EMF) and Cost Tracking in the Logging and Monitoring guide.

CLOUDWATCH_METRICS¶

Purpose : Emit per-request AWS-billed usage as CloudWatch Embedded Metric Format (EMF) log lines

Type : Boolean

Default : false

export CLOUDWATCH_METRICS=true

CLOUDWATCH_METRICS_NAMESPACE¶

Purpose : CloudWatch namespace under which the emitted usage metrics are grouped

Type : String

Default : stdapi

Requirement : 1-255 characters, alphanumeric plus . - _ / # :, must not start with the reserved AWS/ prefix

export CLOUDWATCH_METRICS_NAMESPACE=my-app-metrics

COST_TRACKING¶

Purpose : Enable real-time cost computation from live AWS pricing (details and accuracy caveats). Disabled by default: it requires the extra pricing:GetProducts IAM permission — see Cost Tracking IAM Permissions.

Type : Boolean

Default : false

export COST_TRACKING=true

COST_PRICE_OVERRIDES¶

Purpose : Operator-supplied unit price overrides for models not covered by the AWS Price List API (format and example)

Type : JSON object — keys are model IDs, values are dicts mapping dimension name to price per one unit

Default : {}

Bedrock Guardrails¶

Amazon Bedrock Guardrails add content filtering and safety controls to model inputs and outputs. The configured guardrail also powers the OpenAI-compatible Moderations API (POST /v1/moderations); without one, that API falls back to inline guardrail checks in supported regions, then Amazon Comprehend.

Route Coverage¶

The configured guardrail applies to every route that serves a request directly. The Batch API is the exception: Amazon Bedrock batch inference cannot apply a guardrail, so a batch a configured guardrail would cover is refused rather than run unchecked. Chat routes use the native Bedrock integration; routes whose AWS backend has no guardrail mechanism enforce it through the ApplyGuardrail API: client-supplied text is checked as INPUT before the backend call and generated text as OUTPUT after it. On the Realtime API, where content streams both ways for as long as the session is open, the check runs per turn and an intervention ends the session — see Guardrail coverage for what that does and does not catch.