Configuration Guide¶

stdapi.ai is configured entirely through environment variables, which are read once at startup and cannot be changed without restarting the service. This guide explains each setting category with practical examples to help you configure the service correctly.

What you can configure:

• AWS regions - Access models across multiple regions for availability and model selection
• Data sovereignty - Control which AWS regions are used for compliance (GDPR, HIPAA, etc.)
• Storage - S3 buckets for file operations, regional buckets for multi-region deployments
• Authentication - API keys via SSM or Secrets Manager for secure access control
• Observability - Logging levels, OpenTelemetry, request/response debugging
• Security - CORS, proxy headers, trusted hosts for production deployments
• Performance - Caching, model overrides, S3 acceleration
• TLS / SSL - End-to-end encryption using Granian environment variables

Quick Start¶

For production deployments, configure these essential settings:

Minimal Production Setup¶

Single-region deployment with file storage only.

# S3 bucket for file storage (must be in same region as your server)
export AWS_S3_BUCKET=my-stdapi-bucket

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Production with Authentication¶

Adds secure API key authentication via AWS Systems Manager.

# S3 bucket for file storage (must be in same region as your server)
export AWS_S3_BUCKET=my-stdapi-bucket

# Secure API authentication (recommended: SSM Parameter Store)
export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Full Production Setup (All Features Enabled)¶

Multi-region deployment with all AWS AI services, observability, and security features.

# Core AWS configuration - host server in first region
export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,eu-west-1

# S3 bucket for file storage (must be in us-east-1, your first/primary region)
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket

# Optional: Transcribe S3 bucket (defaults to AWS_S3_BUCKET if not specified)
# Only set this if you need a separate bucket or if transcribe is in a different region
# export AWS_TRANSCRIBE_S3_BUCKET=my-stdapi-transcribe-us-east-1

# Optional: Regional buckets for async/batch inference in other regions
export AWS_S3_REGIONAL_BUCKETS='{"us-west-2": "my-stdapi-us-west-2-bucket", "eu-west-1": "my-stdapi-eu-west-1-bucket"}'

# AWS AI services regions (optional - when unset, every AWS_BEDROCK_REGIONS entry is a
# candidate with automatic failover; set one to pin the service to a single region)
export AWS_POLLY_REGION=us-east-1           # Text-to-speech
export AWS_TRANSCRIBE_REGION=us-east-1      # Speech-to-text (audio transcription)
export AWS_COMPREHEND_REGION=us-east-1      # Language detection & moderation
export AWS_TRANSLATE_REGION=us-east-1       # Text translation

# Authentication
export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

# Logging
export LOG_LEVEL=warning
export LOG_CLIENT_IP=true

# Optional: OpenTelemetry observability (AWS X-Ray integration)
# export OTEL_ENABLED=true
# export OTEL_SERVICE_NAME=stdapi-production
# export OTEL_SAMPLE_RATE=0.1

# Production security settings (when behind AWS ALB/CloudFront)
export ENABLE_PROXY_HEADERS=true

# Note: TRUSTED_HOSTS not recommended with AWS ALB - use ALB host-based routing instead
# Only use TRUSTED_HOSTS if you cannot configure host validation at the load balancer level

# Optional: CORS for browser-based web applications
# export CORS_ALLOW_ORIGINS='["https://app.example.com"]'

Development Setup¶

Local development configuration with API documentation and debug logging enabled.

# Minimal configuration for local development
export AWS_S3_BUCKET=my-stdapi-dev-bucket

# Enable API documentation
export ENABLE_DOCS=true
export ENABLE_REDOC=true

# Full request/response logging for debugging
export LOG_LEVEL=info
export LOG_REQUEST_PARAMS=true

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Environment Variable Summary¶

This section provides a quick reference of all available configuration options. Detailed explanations for each variable can be found in the sections below.

Essential (Production)¶

AWS Client¶

AWS Storage¶

AWS AI Services¶

Resilience & Failover¶

Bedrock Mantle¶

Bedrock Advanced¶

Authentication¶

Configure one source. If several are set, precedence is API_KEY → SSM Parameter Store → Secrets Manager — see Authentication:

API Compatibility¶

Logging¶

CloudWatch Metrics¶

Cost Tracking¶

Observability (OpenTelemetry)¶

HTTP/Security¶

Application Behavior¶

API Documentation¶

MCP (Model Context Protocol)¶

AWS Services and Regions¶

General Configuration¶

AWS_ADAPTIVE_RETRY¶

Purpose : Enable adaptive retry mode that adjusts retry pacing based on observed error rates across all AWS service calls

Type : Boolean (true / false)

Default : false

Behavior : When enabled, the retry strategy dynamically responds to real-time congestion signals. If errors are occurring frequently, retries are spaced further apart to avoid amplifying load on an already-stressed endpoint. Once conditions improve, the pacing returns to normal. When disabled, retries follow a standard exponential backoff strategy with fixed intervals. Applies to all AWS services (Bedrock, S3, Polly, Transcribe, etc.).

# Default: standard exponential backoff
export AWS_ADAPTIVE_RETRY=false

# Enable adaptive retry (recommended under sustained high load)
export AWS_ADAPTIVE_RETRY=true

AWS_MAX_POOL_CONNECTIONS¶

Purpose : Maximum number of concurrent HTTP connections per AWS service client

Type : Integer (must be > 0)

Default : 50

Behavior : Each AWS service client (one per service per region) maintains its own connection pool up to this limit. Under high concurrency, increasing this value prevents requests from queuing for an available connection. Setting it too high may exhaust system file descriptors.

# Default
export AWS_MAX_POOL_CONNECTIONS=50

# High-concurrency deployment
export AWS_MAX_POOL_CONNECTIONS=100

AWS_CONNECT_TIMEOUT¶

Purpose : Timeout in seconds for establishing a connection to an AWS service endpoint

Type : Integer (must be > 0)

Default : 5

Behavior : Limits how long the client waits when opening a new connection. A short value allows fast failover to another region when an endpoint is unreachable. Increase it only if you see spurious connection timeouts on high-latency networks.

# Default: 5 seconds
export AWS_CONNECT_TIMEOUT=5

# High-latency network
export AWS_CONNECT_TIMEOUT=10

Storage Configuration¶

AWS_S3_BUCKET¶

Purpose : Primary S3 bucket for storing generated files (images, audio, documents) and temporary data during processing

Default : None (must be configured for file operations)

Best Practice : The bucket must be in the first region specified in AWS_BEDROCK_REGIONS (your primary region where the server should be hosted) to avoid cross-region data transfer costs and reduce latency

export AWS_S3_BUCKET=my-llm-storage-us-east-1

AWS_S3_ACCELERATE¶

Purpose : Enable S3 Transfer Acceleration for presigned URLs to improve download performance for large files

Type : Boolean

Default : false

Best Practice : Enable when serving large files (high-resolution images, audio) to geographically distributed users

export AWS_S3_ACCELERATE=true

AWS_S3_TMP_PREFIX¶

Purpose : S3 prefix (folder path) for temporary files used during job processing

Default : tmp/

Best Practice : Configure S3 lifecycle policies to automatically delete objects under this prefix after 1 day

export AWS_S3_TMP_PREFIX=tmp/

Custom prefix examples:

# Production environment
export AWS_S3_TMP_PREFIX=prod/tmp/

# Staging environment
export AWS_S3_TMP_PREFIX=staging/tmp/

# Organize by date (requires manual updates)
export AWS_S3_TMP_PREFIX=tmp/2025/01/

# No prefix (store at bucket root - not recommended)
export AWS_S3_TMP_PREFIX=

AWS_S3_FILES_PREFIX¶

Purpose : S3 prefix (folder path) for Files API objects (OpenAI and Anthropic /v1/files endpoints)

Default : files/

Best Practice : Configure an AbortIncompleteMultipartUpload S3 lifecycle rule on this prefix to clean up abandoned upload parts, and apply Intelligent-Tiering for cost optimisation

export AWS_S3_FILES_PREFIX=files/

Custom prefix examples:

# Production environment
export AWS_S3_FILES_PREFIX=prod/files/

# Staging environment
export AWS_S3_FILES_PREFIX=staging/files/

# No prefix (store at bucket root - not recommended)
export AWS_S3_FILES_PREFIX=

AWS_S3_VIDEOS_PREFIX¶

Purpose : S3 prefix (folder path) for videos generated through the Videos API

Default : videos/

Requirement : Must be non-empty, use only S3-safe characters (alphanumerics plus ! _ . * ' ( ) - per path segment), and end with a trailing / — an empty value would widen the ownership check that scopes listing/retrieval to the whole bucket

Best Practice : Generated videos persist until deleted through the API — configure an S3 lifecycle rule on this prefix to cap storage costs

export AWS_S3_VIDEOS_PREFIX=videos/

Amazon Bedrock writes each video generation job's output (MP4 and manifest) under this prefix, in a folder named after the job. Because Amazon Bedrock requires the output bucket to be in the same region as the invocation, videos are stored in the AWS_S3_REGIONAL_BUCKETS bucket of the region that served the job.

AWS_S3_VIDEOS_EXPIRES_AFTER¶

Purpose : Retention period in seconds (minimum 3600) for videos generated through the Videos API

Default : Unset — videos never expire and persist until deleted through the API

Best Practice : Pair with an S3 Lifecycle expiration rule on AWS_S3_VIDEOS_PREFIX covering the same duration (rounded up to whole days) so the objects are actually deleted

# Expire generated videos after 24 hours
export AWS_S3_VIDEOS_EXPIRES_AFTER=86400

When set, the Video object reports expires_at (job completion time plus this value) and downloading expired video content returns a 404. The server enforces expiry at the API level only; the paired S3 Lifecycle rule performs the physical cleanup.

AWS_TRANSCRIBE_S3_BUCKET¶

Purpose : Temporary S3 bucket for transcription workflows

Default : Falls back to AWS_S3_BUCKET if not specified

Requirement : Must be in the same region as AWS_TRANSCRIBE_REGION when that is set; with the default multi-region behavior it serves the primary Bedrock region, and AWS_S3_REGIONAL_BUCKETS entries serve the other candidate regions

# If AWS_TRANSCRIBE_REGION is us-east-1
export AWS_TRANSCRIBE_S3_BUCKET=my-transcribe-temp-us-east-1

# If AWS_TRANSCRIBE_REGION is eu-west-1
export AWS_TRANSCRIBE_S3_BUCKET=my-transcribe-temp-eu-west-1

AWS_S3_REGIONAL_BUCKETS¶

Purpose : Region-specific S3 buckets for Bedrock async and batch inference operations

Default : Empty (no regional buckets configured)

Format : JSON object with region names as keys and bucket names as values

Requirement : Some Bedrock models require S3 buckets in the same region for async and batch inference operations

export AWS_S3_REGIONAL_BUCKETS='{"us-east-1": "my-bedrock-temp-us-east-1", "eu-west-1": "my-bedrock-temp-eu-west-1"}'

AWS_S3_ACCEPTED_BUCKETS¶

Purpose : Declare external S3 buckets that the application has read access to, mapped to their AWS region

Type : JSON object (keys: bucket names, values: AWS region identifiers)

Default : {} (empty — only the application's own buckets are recognized)

Behavior : Declaring a bucket here enables input access to objects the application does not own:

- **S3 URI and S3 HTTP URL access** — `s3://` URIs and S3 HTTP URLs (including presigned URLs) pointing at these buckets are accepted as input sources; an HTTP URL is automatically converted to an `s3://` URI so Bedrock can access the object directly.
- **Declared region** — The region mapped to each bucket is used to reach that bucket in its own region when reading the input object. It does not influence model or inference region selection.

Without this setting, only the application's own buckets (`AWS_S3_BUCKET` and `AWS_S3_REGIONAL_BUCKETS`) are recognized.

export AWS_S3_ACCEPTED_BUCKETS='{"my-data-bucket": "us-east-1", "my-eu-bucket": "eu-west-1"}'

{
  "Effect": "Allow",
  "Action": "s3:GetObject",
  "Resource": [
    "arn:aws:s3:::my-data-bucket/*",
    "arn:aws:s3:::my-eu-bucket/*"
  ]
}

S3 Bucket Lifecycle Configuration¶

Purpose : Configure automatic deletion of temporary files and abandoned multipart upload parts to minimize storage costs

Recommendation : Configure S3 lifecycle policies to automatically delete objects under the AWS_S3_TMP_PREFIX after 1 day, and abort incomplete multipart uploads under the AWS_S3_FILES_PREFIX after 1 day

stdapi.ai stores temporary files under the prefix configured by AWS_S3_TMP_PREFIX (default: tmp/). These include generated images, audio files, and transcription workflow files. Configure S3 lifecycle policies to automatically delete objects under this prefix after 1 day.

Additionally, multipart file uploads (OpenAI Uploads API) store parts under AWS_S3_FILES_PREFIX (default: files/). If a session is never completed or cancelled — for example when a client disconnects — the uploaded parts remain in S3 and accumulate costs. Add an AbortIncompleteMultipartUpload rule on the files prefix to clean these up automatically.

{
  "Rules": [
    {
      "Id": "DeleteTemporaryFiles",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "tmp/"
      },
      "Expiration": {
        "Days": 1
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 1
      }
    },
    {
      "Id": "AbortIncompleteMultipartUploads",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "files/"
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 1
      }
    }
  ]
}

Apply via AWS CLI:

# For primary S3 bucket (AWS_S3_BUCKET)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-stdapi-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

# For transcribe S3 bucket (AWS_TRANSCRIBE_S3_BUCKET, if different from AWS_S3_BUCKET)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-transcribe-temp-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

# For regional buckets (AWS_S3_REGIONAL_BUCKETS)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-stdapi-us-west-2-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

Bedrock Configuration¶

AWS_BEDROCK_REGIONS¶

Purpose : List of AWS regions where Bedrock models are available

Format : Comma-separated string

Default : Current AWS SDK region if not specified

Behavior : Models are discovered in the same order as the listed regions. The first region is the primary region where your server should be hosted on AWS for optimal performance. Your S3 bucket (AWS_S3_BUCKET) must also be in this region. If a model is unavailable in the primary region, subsequent regions are checked in order

export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,eu-west-1

AWS_BEDROCK_CROSS_REGION_INFERENCE¶

Purpose : Enable automatic cross-region routing when a model isn't available in the primary region

Type : Boolean

Default : true

export AWS_BEDROCK_CROSS_REGION_INFERENCE=true

AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL¶

Purpose : Allow global cross-region inference routing to any region worldwide

Type : Boolean

Default : true

export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

AWS_BEDROCK_REGION_ROUTING¶

Purpose : Automatic region routing strategy for distributing Bedrock requests across configured regions

Type : String

Default : ordered

Behavior : When multiple regions are configured in AWS_BEDROCK_REGIONS, this setting controls how requests are distributed across them. The router automatically handles quota/throttling errors and regional unavailability by temporarily avoiding affected regions

Requirement : Requires at least 2 regions in AWS_BEDROCK_REGIONS to take effect

Available strategies:

# Use ordered routing (default)
export AWS_BEDROCK_REGION_ROUTING=ordered

# Use lowest latency routing
export AWS_BEDROCK_REGION_ROUTING=lowest_latency

# Disable routing
export AWS_BEDROCK_REGION_ROUTING=disabled

AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS¶

Purpose : Duration to temporarily avoid a region after receiving a quota or throttling error

Type : Integer (seconds, must be > 0)

Default : 60

Behavior : This is the base backoff value. When a Bedrock API call fails due to quota limits (ThrottlingException, TooManyRequestsException, ServiceQuotaExceededException), the affected region is temporarily blocked. The actual delay doubles with each consecutive quota error on the same region (exponential backoff), up to a hard ceiling of 1 hour. The counter resets after a successful request. Subsequent requests are routed to other available regions during the backoff period.

# Default: 60 seconds (base value — actual delay doubles per consecutive error)
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=60

# Shorter base backoff for aggressive retry
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=30

# Longer base backoff for conservative approach
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=120

AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS¶

Purpose : Duration to temporarily avoid a region after receiving an unavailability error

Type : Integer (seconds, must be > 0)

Default : 30

Behavior : When a Bedrock API call fails due to service unavailability (ServiceUnavailableException, ModelNotReadyException), the affected region is temporarily blocked for this many seconds. These errors are typically shorter-lived than quota limits, so the default is shorter

# Default: 30 seconds
export AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS=30

# Longer backoff for stability
export AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS=60

AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS¶

Purpose : Hard ceiling in seconds on the exponential quota backoff for a single region

Type : Integer (seconds, must be > 0)

Default : 3600 (1 hour)

Behavior : Quota backoff grows exponentially with consecutive errors (base interval × 2^n). This setting caps how large that value can become, preventing a region from being blocked indefinitely. Reduce it to allow faster recovery; increase it to keep a misbehaving region sidelined for longer.

# Default: 1 hour ceiling
export AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS=3600

# More aggressive recovery
export AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS=600

AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR¶

Purpose : Multiplier applied to the max quota backoff to compute the stale-error reset threshold

Type : Integer (must be > 0)

Default : 2 (threshold = 2 × max quota backoff = 2 hours with defaults)

Behavior : If the most recent quota error on a region occurred more than max_quota_backoff × factor seconds ago, the consecutive-error counter is reset and the next error is treated as a fresh start rather than an escalation. A higher value keeps memory of past errors for longer before resetting the counter.

# Default: reset counter after 2× the max backoff window
export AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR=2

# Longer memory of past errors
export AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR=4

AWS_BEDROCK_MAX_RETRIES¶

Purpose : Maximum number of retries per Bedrock invocation, each retry escalating to the next available region

Type : Integer (must be 0 or greater; 0 disables retries)

Default : 9

Behavior : Controls the retry budget for each Bedrock API call. When region routing is enabled, every retry escalates to the next region in priority order and each candidate region is tried at most once, so the attempts are bounded by the smaller of AWS_BEDROCK_MAX_RETRIES + 1 and the number of candidate regions for the model — with 3 regions and the default 9 retries, a request makes at most 3 attempts. A region that just failed is still blocked by its own backoff, and retrying it would only extend that backoff instead of recovering the request. When routing is disabled, or the region is pinned by S3 inputs, the full budget is spent as SDK retries against that single region.

# Default: 9 retries (10 total attempts)
export AWS_BEDROCK_MAX_RETRIES=9

# Fail faster (e.g. low-latency interactive use cases)
export AWS_BEDROCK_MAX_RETRIES=3

# Deeper in-region retrying for single-region or S3-pinned requests
export AWS_BEDROCK_MAX_RETRIES=18

AWS_FAILOVER_MAX_RETRIES¶

Purpose : Maximum SDK retry attempts per candidate region for the multi-region failover services (Polly, Transcribe, Translate, Comprehend)

Type : Integer (must be 0 or greater)

Default : 2

Behavior : Only applied when a service has several candidate regions (no explicit region setting): each region attempt uses this reduced retry budget (2 retries = 3 attempts per region) before failing over, so failover across regions replaces deep in-region retrying. When a service is pinned to a single region, the standard retry budget from AWS_BEDROCK_MAX_RETRIES applies instead.

# Default: 2 retries (3 attempts) per candidate region
export AWS_FAILOVER_MAX_RETRIES=2

# Fail over after a single attempt per region
export AWS_FAILOVER_MAX_RETRIES=0

AWS_BEDROCK_MANTLE_ENABLED¶

Purpose : Expose models served by the Amazon Bedrock Mantle endpoint (OpenAI/Anthropic-compatible APIs) in addition to the classic Bedrock Converse models

Type : Boolean

Default : true

Behavior : Mantle-only models (e.g. OpenAI GPT, xAI Grok, Google Gemma 4) become available on the chat completions, responses, messages, and completions routes. Models available on both the classic bedrock-runtime endpoint and Mantle are served by bedrock-runtime unless listed in AWS_BEDROCK_MANTLE_PREFERRED_MODELS.

Authentication requires no static secrets: short-term bearer tokens are derived automatically (SigV4-presigned) from the same AWS credential chain the server already uses, and refreshed transparently.

When Bedrock Mantle is unreachable or the IAM role lacks `bedrock-mantle` permissions, Mantle models are simply not listed and a warning is logged at startup — no configuration change required.

export AWS_BEDROCK_MANTLE_ENABLED=false

Bedrock Mantle Models feature overview

AWS_BEDROCK_MANTLE_REGIONS¶

Purpose : List of AWS regions used for Amazon Bedrock Mantle, in failover priority order

Type : Comma-separated string of AWS region identifiers

Default : AWS_BEDROCK_REGIONS when unset

Behavior : Model availability differs per region; the served model catalog is the union of all listed regions. Region failover, quota backoff, and health tracking work exactly like classic Bedrock region routing.

export AWS_BEDROCK_MANTLE_REGIONS=us-east-1,eu-west-1

AWS_BEDROCK_MANTLE_ENDPOINT_URL¶

Purpose : Override the Amazon Bedrock Mantle endpoint URL template

Type : String — URL template with a {region} placeholder

Default : None (https://bedrock-mantle.{region}.api.aws)

Behavior : The {region} placeholder is substituted with the target region.

export AWS_BEDROCK_MANTLE_ENDPOINT_URL='https://bedrock-mantle.{region}.api.aws'

AWS_BEDROCK_MANTLE_PREFERRED_MODELS¶

Purpose : Model IDs (or ID prefixes) served by Amazon Bedrock Mantle even when also available on the classic bedrock-runtime endpoint

Type : Comma-separated string of model IDs or ID prefixes

Default : [] (empty — dual-homed models are served by bedrock-runtime)

Behavior : Useful to leverage Mantle's independent throughput quotas or native response storage for selected models. Mantle quotas (per-model, per-region tokens-per-minute) are independent from bedrock-runtime quotas.

export AWS_BEDROCK_MANTLE_PREFERRED_MODELS='anthropic.claude-haiku-4-5,openai.gpt-oss'

AWS_BEDROCK_MANTLE_SERVICE_HEADER¶

Purpose : Honor the x-stdapi-service: bedrock-mantle request header to route a model available on both endpoints through Bedrock Mantle for that request instead of the default bedrock-runtime serving

Type : Boolean

Default : false

export AWS_BEDROCK_MANTLE_SERVICE_HEADER=true

AWS_BEDROCK_MANTLE_PROJECT¶

Purpose : Default Amazon Bedrock Project/Workspace ID attributed to Bedrock Mantle inference requests for cost tracking and observability

Type : String — a bare project ID (e.g. proj_abc123 or default), not an ARN

Default : None (requests fall to the account's default project)

Behavior : Bedrock Projects (OpenAI-compatible APIs) and Workspaces (Anthropic Messages API) are the same underlying resource; the value is sent as the OpenAI-Project header on the Chat Completions and Responses APIs, and as the anthropic-workspace header on the Anthropic Messages API. When unset, requests fall to the account's default project — no failure.

export AWS_BEDROCK_MANTLE_PROJECT=proj_abc123

AWS_BEDROCK_ALLOW_MANTLE_PROJECT_OVERRIDE¶

Purpose : Allow a request to override the configured Mantle project via the OpenAI-Project / anthropic-workspace header

Type : Boolean

Default : false

Behavior : When true, a request may set its own project through the OpenAI-Project (Chat Completions, Responses) or anthropic-workspace (Anthropic Messages) header. When false and AWS_BEDROCK_MANTLE_PROJECT is configured, the request header is ignored and the server default applies. When no default project is configured, the request header is always honored regardless of this flag. A malformed request-supplied project ID returns 400.

export AWS_BEDROCK_ALLOW_MANTLE_PROJECT_OVERRIDE=true

AWS_BEDROCK_MODEL_REGION_RESTRICT¶

Purpose : Restrict a model to specific region(s) only, useful when a model provides important features only in certain regions

Type : JSON object (keys: Bedrock model IDs or prefixes, values: ordered lists of allowed regions)

Default : {} (empty — no model-specific region restriction)

Behavior : When set, the model is made available only in the listed regions (intersected with the regions where it is actually available), and the list order defines the routing priority when the default ordered routing strategy is used. No fallback to other regions occurs. Keys can be exact model IDs or prefixes that match the beginning of a model ID

# Restrict Nova Pro to us-east-1 for grounding support
export AWS_BEDROCK_MODEL_REGION_RESTRICT='{"amazon.nova-pro-v1:0": ["us-east-1"]}'

AWS_BEDROCK_LEGACY¶

Purpose : Allow usage of legacy/deprecated Bedrock models

Type : Boolean

Default : false

export AWS_BEDROCK_LEGACY=true

AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK¶

Purpose : Transparently reroute requests using a deprecated model ID to its recommended replacement

Type : Boolean

Default : true

Behavior : When true, any request that specifies a deprecated model ID (as listed in the server's deprecation registry) is silently retried with the recommended replacement model. The replacement is fully re-evaluated — alias resolution, modality checks, and region routing all apply to the new model ID. When false, deprecated model IDs return a 404 error with a message indicating the replacement, forcing clients to migrate explicitly.

# Transparent fallback (default) — clients using old model IDs keep working
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=true

# Strict mode — deprecated model IDs return 404, clients must update their code
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=false

AWS_BEDROCK_DEPRECATED_MODELS¶

Purpose : Extend or override the built-in deprecated model registry with custom mappings

Type : JSON object — dict[str, str]

Default : {}

Behavior : Merged with the built-in registry at startup. User-provided entries take precedence over built-in ones — this means it can be used both to add new deprecated model mappings and to override the fallback target of an already-defined deprecated model. Effective only when AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK is true.

Reference : Amazon Bedrock model lifecycle

# Add a custom deprecated model and override an existing built-in mapping
export AWS_BEDROCK_DEPRECATED_MODELS='{"my-old-model-v1": "my-new-model-v2", "amazon.titan-text-lite-v1": "amazon.nova-lite-v1:0"}'

AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE¶

Purpose : Control automatic subscription to new models in AWS Marketplace

Type : Boolean

Default : true

Behavior : When true, the server automatically subscribes to new models discovered in the AWS Marketplace, making them immediately available through the API. When false, only models with existing marketplace subscriptions are visible and accessible

IAM Permissions Required : aws-marketplace:Subscribe, aws-marketplace:ViewSubscriptions — see Marketplace Auto-Subscribe IAM

# Allow automatic subscription (default)
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=true

# Restrict to pre-subscribed models only
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=false

AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass cross-region inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use cross-region inference profile ARNs instead of model IDs in the model parameter. Cross-region inference profiles enable routing to multiple regions for better availability

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable cross-region inference profile ARN support
export AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-sonnet-5

AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass application inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use application inference profile ARNs instead of model IDs in the model parameter. Application inference profiles are custom routing configurations for specific use cases

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable application inference profile ARN support
export AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123xyz

AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN¶

Purpose : Allow users to pass prompt router ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use prompt router ARNs instead of model IDs in the model parameter. Prompt routers enable dynamic model selection based on prompt characteristics

IAM Permissions Required : bedrock:GetPromptRouter (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable prompt router ARN support
export AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN=true

arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router

AWS_BEDROCK_ALLOW_PROMPT_ARN¶

Purpose : Allow users to reference an Amazon Bedrock Prompt Management prompt ARN in the OpenAI Responses API prompt parameter

Type : Boolean

Default : false

Behavior : When enabled, prompt.id accepts a prompt ARN (with an optional prompt.version) and prompt.variables fill in the template. Amazon Bedrock renders the stored prompt, and the model it is bound to serves the request. When disabled, any prompt parameter is rejected with a 400 error

IAM Permissions Required : bedrock:GetPrompt (resolve the prompt's model) and bedrock:RenderPrompt (invoke it)

# Disabled (default) - the Responses API `prompt` parameter returns 400
# No environment variable needed

# Enable Prompt Management prompt ARN support
export AWS_BEDROCK_ALLOW_PROMPT_ARN=true

arn:aws:bedrock:us-east-1:123456789012:prompt/ABCDE12345:1

AWS_BEDROCK_MODEL_ARN_MAPPING¶

Purpose : Map standard model IDs to custom inference profile or prompt router ARNs for server-controlled routing

Format : JSON object with model IDs as keys and ARNs as values

Default : {} (empty, no mappings)

Behavior : When configured, the mapped ARN is used instead of the default cross-region inference profile when clients request the model by its standard ID. This provides centralized control over model routing without requiring client changes

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-custom-profile",
  "anthropic.claude-haiku-4-5-20251001-v1:0": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/cost-optimizer"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/production-profile"
}'

# Production: Use cost-optimized prompt router
export AWS_BEDROCK_MODEL_ARN_MAPPING='{"anthropic.claude-sonnet-5": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/prod-router"}'

# Development: Use standard cross-region profile
export AWS_BEDROCK_MODEL_ARN_MAPPING='{}'

Other AWS Services¶

AWS_POLLY_REGION¶

Purpose : Region for Amazon Polly text-to-speech service

Default : All regions in AWS_BEDROCK_REGIONS, with per-engine regional discovery and automatic failover

Behavior : When unset, voice availability is discovered per engine in every AWS_BEDROCK_REGIONS entry at startup: an engine (Standard, Neural, Long-form, Generative) is exposed as a model when at least one candidate region offers it, and each synthesis call routes to the regions offering the requested engine and voice, failing over on region-level errors. Setting an explicit region pins Polly to that single region — engines it does not offer are then disabled.

export AWS_POLLY_REGION=us-east-1

AWS_COMPREHEND_REGION¶

Purpose : Region for the Amazon Comprehend services (language detection and toxicity moderation)

Default : All regions in AWS_BEDROCK_REGIONS, tried in order with automatic failover

Behavior : When unset, Comprehend calls try each AWS_BEDROCK_REGIONS entry in order and fail over to the next region on region-level errors (throttling, service unavailability, network issues, or a region that does not offer Comprehend or the requested operation). Setting an explicit region pins Comprehend to that single region with no failover.

export AWS_COMPREHEND_REGION=us-east-1

AWS_TRANSCRIBE_REGION¶

Purpose : Region for Amazon Transcribe speech-to-text service

Default : All regions in AWS_BEDROCK_REGIONS that have a co-located S3 bucket, tried in order with automatic failover

Behavior : Transcription jobs need an S3 bucket in the job's region. When unset, every AWS_BEDROCK_REGIONS entry with a usable bucket is a candidate — the primary region is served by AWS_TRANSCRIBE_S3_BUCKET (or AWS_S3_BUCKET), the others by their AWS_S3_REGIONAL_BUCKETS entry. On a region-level error while starting a job, the audio is server-side copied to the next candidate's bucket and the job restarts there. Setting an explicit region pins Transcribe to that single region with no failover.

export AWS_TRANSCRIBE_REGION=us-east-1

AWS_TRANSLATE_REGION¶

Purpose : Region for Amazon Translate text translation service

Default : All regions in AWS_BEDROCK_REGIONS, tried in order with automatic failover

Behavior : When unset, translation calls try each AWS_BEDROCK_REGIONS entry in order and fail over to the next region on region-level errors (throttling, service unavailability, network issues, or a region that does not offer Translate). Setting an explicit region pins Translate to that single region with no failover.

export AWS_TRANSLATE_REGION=us-east-1

Compliance and Latency Optimization¶

Strategic region configuration is critical for both regulatory compliance and performance optimization. This section provides best practice configurations for common scenarios.

GDPR and Data Residency Compliance¶

For applications serving European users, data residency regulations like GDPR may require that data processing occurs within specific geographic boundaries.

# Use only European regions
export AWS_S3_BUCKET=my-stdapi-eu-bucket
export AWS_BEDROCK_REGIONS=eu-west-1,eu-west-3,eu-central-1

# Disable global cross-region inference to prevent data routing outside Europe
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

# Keep cross-region inference enabled for failover within EU regions
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true

Latency Optimization¶

For applications prioritizing low latency and high performance, configure regions closest to your users and application infrastructure.

North America:

# Primary region for lowest latency, with fallbacks
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket
export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,us-east-2

# Enable all cross-region inference for maximum model availability
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Asia-Pacific:

# Use Asia-Pacific regions for lowest latency to APAC users
export AWS_S3_BUCKET=my-stdapi-ap-southeast-1-bucket
export AWS_BEDROCK_REGIONS=ap-southeast-1,ap-northeast-1,us-west-2

# Enable global inference for fallback to US regions if needed
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Global Multi-Region:

# Balanced configuration with worldwide coverage
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket
export AWS_BEDROCK_REGIONS=us-east-1,eu-west-1,ap-southeast-1,us-west-2

# Enable global inference for best availability
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Hybrid Approach: Compliance with Performance¶

Balance compliance requirements with performance needs:

# EU primary with US fallback (for model availability)
export AWS_S3_BUCKET=my-stdapi-eu-bucket
export AWS_BEDROCK_REGIONS=eu-west-1,eu-central-1,us-east-1

# Allow cross-region but restrict to specific regions only
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

Configuration Order¶

When deploying stdapi.ai, configure settings in this recommended order:

1. IAM Permissions - Set up AWS access first
2. AWS Services and Regions - Configure S3 buckets and Bedrock regions
3. Authentication - Secure your API with authentication
4. Optional Features - Add observability, guardrails, and other features as needed

IAM Permissions¶

The full IAM reference — required Amazon Bedrock permissions, per-feature policy statements, complete policy examples, and AWS tag policy requirements — has moved to the dedicated IAM Permissions page.

Authentication¶

stdapi.ai supports three methods for API key authentication.

Method 1: Direct API Key¶

Provide the API key directly via environment variable. Intended for local development and testing; it takes precedence over both AWS-backed sources.

API_KEY¶

Purpose : Static API key value

Security Warning : Avoid hardcoding in configuration files; use environment variables only

Client Usage : Clients must include this key in the Authorization: Bearer <key> header or X-API-Key header

export API_KEY=sk-1234567890abcdef...

Method 2: SSM Parameter Store (Recommended)¶

Recommended - Use AWS Systems Manager Parameter Store for secure key storage with encryption, access control, and auditing. This method should be used only with already existing parameters.

API_KEY_SSM_PARAMETER¶

Purpose : Name of the SSM parameter containing the API key. The parameter is retrieved from the current region detected by the running container, or defaults to the first region in AWS_BEDROCK_REGIONS.

Recommendation : Use SecureString type for encryption at rest

IAM Permissions Required : ssm:GetParameter, kms:Decrypt (if encrypted)

export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

Method 3: Secrets Manager¶

Use AWS Secrets Manager for secure key storage with automatic rotation support. This method should be used only with already existing secrets.

API_KEY_SECRETSMANAGER_SECRET¶

Purpose : Name of the Secrets Manager secret containing the API key. The secret is retrieved from the current region detected by the running container, or defaults to the first region in AWS_BEDROCK_REGIONS.

Format : Can be a plain string or JSON object

IAM Permissions Required : secretsmanager:GetSecretValue

API_KEY_SECRETSMANAGER_KEY¶

Purpose : JSON key name within the secret (if the secret is a JSON object)

Default : api_key

Plain String Secret:

export API_KEY_SECRETSMANAGER_SECRET=stdapi-api-key

JSON Secret:

export API_KEY_SECRETSMANAGER_SECRET=stdapi-credentials
export API_KEY_SECRETSMANAGER_KEY=api_key

Example JSON secret structure:

{
  "api_key": "sk-1234567890abcdef...",
  "other_config": "value"
}

API Compatibility¶

Configure the base URL paths for OpenAI and Anthropic-compatible API routes.

OPENAI_ROUTES_PREFIX¶

Purpose : Base path prefix for OpenAI-compatible API routes

Default : `` (empty, routes mounted at root)

Requirement : Empty, or a path starting with / with no trailing slash, using only alphanumeric characters and . _ ~ - per segment; must differ from ANTHROPIC_ROUTES_PREFIX and COHERE_ROUTES_PREFIX

Effect : All OpenAI-compatible endpoints will be mounted under this prefix

export OPENAI_ROUTES_PREFIX=/api

ANTHROPIC_ROUTES_PREFIX¶

Purpose : Base path prefix for Anthropic-compatible API routes

Default : /anthropic

Requirement : A path starting with / with no trailing slash, using only alphanumeric characters and . _ ~ - per segment; must differ from OPENAI_ROUTES_PREFIX and COHERE_ROUTES_PREFIX

Effect : All Anthropic-compatible endpoints will be mounted under this prefix

export ANTHROPIC_ROUTES_PREFIX=/anthropic

export ANTHROPIC_ROUTES_PREFIX=/api/anthropic

COHERE_ROUTES_PREFIX¶

Purpose : Base path prefix for Cohere-compatible API routes

Default : /cohere

Requirement : A path starting with / with no trailing slash, using only alphanumeric characters and . _ ~ - per segment; must differ from OPENAI_ROUTES_PREFIX and ANTHROPIC_ROUTES_PREFIX

Effect : All Cohere-compatible endpoints will be mounted under this prefix

export COHERE_ROUTES_PREFIX=/cohere

CORS Configuration¶

Configure Cross-Origin Resource Sharing (CORS) to control which web origins can access your API from browsers.

CORS_ALLOW_ORIGINS¶

Purpose : List of origins allowed to make cross-origin requests

Format : JSON array of origin URLs

Default : None (CORS not enabled)

Best Practice : Only enable if your API is accessed from web browsers; specify exact origins in production

# Not configured (default) - CORS middleware not enabled
# Browser cross-origin requests will be blocked
# No environment variable needed

# Development: Allow all origins
export CORS_ALLOW_ORIGINS='["*"]'

# Production: Specific origins only
export CORS_ALLOW_ORIGINS='["https://myapp.com", "https://app.example.com"]'

# Multiple environments
export CORS_ALLOW_ORIGINS='["https://app.example.com", "https://staging.example.com"]'

Trusted Host Configuration¶

Configure Host header validation to protect against Host header injection attacks.

TRUSTED_HOSTS¶

Purpose : List of trusted Host header values for validation

Format : JSON array of hostnames (supports wildcards)

Default : None (no Host header validation)

Best Practice : Use AWS ALB host-based routing rules instead when possible for better performance and management

# Production: Specific hosts only
export TRUSTED_HOSTS='["api.example.com", "www.example.com"]'

export TRUSTED_HOSTS='["*.example.com", "*.myapp.com", "api.production.com"]'

export TRUSTED_HOSTS='["api.example.com", "localhost", "127.0.0.1"]'

aws elbv2 create-rule \
  --listener-arn arn:aws:elasticloadbalancing:... \
  --priority 1 \
  --conditions Field=host-header,Values=api.example.com \
  --actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:...

Proxy Headers Configuration¶

Configure X-Forwarded-* header processing when running behind reverse proxies or load balancers.

ENABLE_PROXY_HEADERS¶

Purpose : Enable trusting X-Forwarded-* headers from reverse proxies

Type : Boolean

Default : false (disabled)

Best Practice : Only enable when running behind a trusted reverse proxy

# Disabled (default) - do not trust X-Forwarded-* headers
# No environment variable needed

# Enable when behind reverse proxy
export ENABLE_PROXY_HEADERS=true

# Do NOT enable proxy headers
# ENABLE_PROXY_HEADERS should remain false (default)

export ENABLE_PROXY_HEADERS=true

export ENABLE_PROXY_HEADERS=true

PROXY_TRUSTED_HOSTS¶

Purpose : Restrict which peer IPs may set trusted X-Forwarded-* headers when ENABLE_PROXY_HEADERS is enabled

Type : JSON array of IPs/CIDRs, or *

Default : * (trust every peer — backward compatible)

Best Practice : Restrict to your reverse proxy's IP range so direct clients cannot spoof X-Forwarded-For

# Trust forwarded headers only from the VPC / proxy range
export ENABLE_PROXY_HEADERS=true
export PROXY_TRUSTED_HOSTS='["10.0.0.0/8"]'

TLS / SSL Configuration¶

Configure end-to-end TLS encryption within the container. These are native Granian environment variables and are available with the provided container images.

GRANIAN_SSL_CERTIFICATE¶

Purpose : Path to the SSL certificate file

Type : File path

GRANIAN_SSL_KEYFILE¶

Purpose : Path to the SSL private key file (PKCS#8 format only)

Type : File path

GRANIAN_SSL_KEYFILE_PASSWORD¶

Purpose : Password for the private key file

Type : String

GRANIAN_SSL_PROTOCOL_MIN¶

Purpose : Minimum supported TLS version (tls1.2 or tls1.3)

Type : Enum

Default : tls1.3

GRANIAN_SSL_CA¶

Purpose : Path to the CA certificate bundle used to verify client certificates (mTLS)

Type : File path

GRANIAN_SSL_CLIENT_VERIFY¶

Purpose : Enable client certificate verification (mTLS)

Type : Boolean

Default : false (disabled)

GZip Compression¶

Configure automatic GZip compression for HTTP responses to reduce bandwidth usage and improve response times.

ENABLE_GZIP¶

Purpose : Enable GZip compression for HTTP responses

Type : Boolean

Default : false (disabled)

Best Practice : Use AWS ALB or CloudFront compression instead when available for better performance

# Disabled (default) - no response compression
# No environment variable needed

# Enable GZip compression (responses larger than 1 KiB will be compressed)
export ENABLE_GZIP=true

MCP (Model Context Protocol)¶

When enabled, stdapi.ai exposes its API endpoints as MCP tools, allowing AI clients and agents to call them directly using the Model Context Protocol. The full list of available tool names is documented in API Overview → MCP Tools.

Both transport types can be enabled independently or simultaneously.

ENABLE_MCP_STREAMABLE_HTTP¶

Purpose : Enable the MCP server using Streamable HTTP transport — the recommended method

Type : Boolean

Default : false

Behavior : Exposes an MCP-compatible endpoint at /mcp. AI clients connect using standard HTTP requests following the MCP Streamable HTTP specification.

# Disabled (default)
# No environment variable needed

# Enable MCP Streamable HTTP transport
export ENABLE_MCP_STREAMABLE_HTTP=true

MCP_STATELESS_HTTP¶

Purpose : Serve the Streamable HTTP transport without server-side sessions

Type : Boolean

Default : false

Behavior : Each request to /mcp is handled by a fresh transport that keeps no state. Clients may call tools/list and tools/call without an initialize handshake, an Mcp-Session-Id the server never issued is accepted rather than rejected, and any replica may serve any request.

Requires : ENABLE_MCP_STREAMABLE_HTTP=true. Ignored otherwise.

# Sessions enabled (default)
# No environment variable needed

# Stateless transport
export ENABLE_MCP_STREAMABLE_HTTP=true
export MCP_STATELESS_HTTP=true

ENABLE_MCP_SSE¶

Purpose : Enable the MCP server using Server-Sent Events (SSE) transport

Type : Boolean

Default : false

Behavior : Exposes MCP endpoints at /sse for AI clients that require the SSE transport protocol.

# Disabled (default)
# No environment variable needed

# Enable MCP SSE transport
export ENABLE_MCP_SSE=true

export ENABLE_MCP_STREAMABLE_HTTP=true
export ENABLE_MCP_SSE=true

MCP_INCLUDE_TOOLS¶

Purpose : Expose only a specific subset of MCP tools; all others are hidden

Format : Comma-separated list of tool names (duplicates are automatically removed)

Default : None (all tools exposed)

# All tools exposed by default
# No environment variable needed

# Expose only specific tools
export MCP_INCLUDE_TOOLS="openai_chat_completion,openai_embedding,search_models"

# When both MCP_INCLUDE_TOOLS and MCP_EXCLUDE_TOOLS are specified,
# tools in MCP_EXCLUDE_TOOLS are removed from MCP_INCLUDE_TOOLS:
export MCP_INCLUDE_TOOLS="openai_chat_completion,openai_embedding,search_models"
export MCP_EXCLUDE_TOOLS="openai_files_delete,anthropic_files_delete"
# Result: only openai_chat_completion, openai_embedding, search_models are exposed

See API Overview → MCP Tools for the full list of available tool names.

MCP_EXCLUDE_TOOLS¶

Purpose : Hide specific MCP tools from clients; all others remain exposed

Format : Comma-separated list of tool names (duplicates are automatically removed)

Default : None (no tools excluded)

# No tools excluded by default
# No environment variable needed

# Exclude destructive tools
export MCP_EXCLUDE_TOOLS="openai_files_delete,anthropic_files_delete"

See API Overview → MCP Tools for the full list of available tool names.

Tool Selection Best Practices¶

stdapi.ai exposes a fixed set of tools derived from its API surface — you can include or exclude them by name, but cannot modify or rename them. See API Overview → MCP Tools for the full catalog.

Start from the minimum, not the maximum

By default all tools are exposed. It is safer and more effective to begin with a narrow MCP_INCLUDE_TOOLS list covering only what the workflow needs, then expand it deliberately. LLMs perform better with fewer choices, and many AI providers cap the number of active tools per session.

Always include search_models for agent model discovery

search_models is the recommended tool for agents to discover available model IDs — it supports capability-based filtering (by modality, route, region, streaming support) and returns richer metadata than openai_model_list or anthropic_model_list. Include it in every agent configuration so the agent can resolve the right model dynamically rather than relying on hardcoded IDs:

export MCP_INCLUDE_TOOLS="openai_chat_completion,search_models,openai_embedding"

Always exclude file deletion tools unless required

Uploaded files are the only durable, stateful data managed by stdapi.ai — deletion is permanent and cannot be undone. Unless your workflow explicitly needs to delete files, always suppress these tools:

export MCP_EXCLUDE_TOOLS="openai_files_delete,anthropic_files_delete"

Exclude high-cost tools unless the workflow requires them

Image generation (openai_image_generation, openai_image_edit, openai_image_variation) and speech synthesis (openai_audio_speech) incur a per-call cost that accumulates quickly if an agent invokes them speculatively. Only include them when the use case calls for it and the agent's decision to generate images or audio is intentional.

Use MCP_INCLUDE_TOOLS for the tightest control

For predictable, well-defined workflows, listing tools explicitly with MCP_INCLUDE_TOOLS is more reliable than maintaining an exclusion list. For example, a workflow limited to text generation and model discovery needs only:

export MCP_INCLUDE_TOOLS="openai_chat_completion,search_models"

SSRF Protection¶

Configure Server-Side Request Forgery (SSRF) protection to prevent unauthorized access to internal networks.

SSRF_PROTECTION_BLOCK_PRIVATE_NETWORKS¶

Purpose : Enable SSRF protection by blocking requests to private/local networks

Type : Boolean

Default : true (enabled for security)

Best Practice : Keep enabled in production to protect against SSRF attacks

# Enabled (default) - block private networks
# No environment variable needed

# Disable only in controlled environments that need local network access
export SSRF_PROTECTION_BLOCK_PRIVATE_NETWORKS=false

Request Limits¶

Bound per-request resource usage to protect the server when the API is exposed to untrusted clients.

MAX_INPUT_FILE_SIZE¶

Purpose : Cap the size of an inline input file loaded into memory to protect against memory-exhaustion (DoS)

Type : Integer (bytes)

Default : 0 (disabled — no limit)

Best Practice : Set a limit aligned with your largest expected inline input (e.g. 26214400 for 25 MiB) when the API is exposed to untrusted clients

# Disabled (default) - no size limit
# No environment variable needed

# Reject inline inputs larger than 25 MiB
export MAX_INPUT_FILE_SIZE=26214400

MAX_CONCURRENT_INPUT_DOWNLOADS¶

Purpose : Bound the number of input files fetched or resolved concurrently within a single request

Type : Integer (> 0)

Default : 8

Best Practice : Keep a modest value so a single request with many remote inputs cannot exhaust sockets/memory or amplify outbound requests against a target

# Allow up to 4 concurrent input downloads per request
export MAX_CONCURRENT_INPUT_DOWNLOADS=4

Observability (OpenTelemetry)¶

Configure distributed tracing for debugging and performance monitoring. stdapi.ai integrates with AWS X-Ray, Jaeger, DataDog, and other OTLP-compatible systems.

OTEL_ENABLED¶

Purpose : Enable or disable OpenTelemetry tracing

Type : Boolean

Default : false

export OTEL_ENABLED=true

OTEL_SERVICE_NAME¶

Purpose : Service identifier in trace visualizations

Default : stdapi.ai

Best Practice : Use descriptive names with environment information

export OTEL_SERVICE_NAME=stdapi-production-us-east-1

OTEL_EXPORTER_ENDPOINT¶

Purpose : OTLP HTTP endpoint URL for sending traces

Default : http://127.0.0.1:4318/v1/traces

Protocol : Must support OTLP HTTP format

AWS X-Ray (via ADOT):

export OTEL_EXPORTER_ENDPOINT=http://127.0.0.1:4318/v1/traces

Jaeger:

export OTEL_EXPORTER_ENDPOINT=http://jaeger:14268/api/traces

Cloud Provider OTLP:

# Use provider-specific OTLP endpoints
export OTEL_EXPORTER_ENDPOINT=https://your-provider-otlp-endpoint.com/v1/traces

OTEL_SAMPLE_RATE¶

Purpose : Percentage of requests to trace (controls cost vs. observability)

Type : Float (0.0 to 1.0)

Default : 1.0 (100%)

Development:

# Trace everything for debugging
export OTEL_SAMPLE_RATE=1.0

Production (Moderate Traffic):

# Sample 10% of requests
export OTEL_SAMPLE_RATE=0.1

Production (High Traffic):

# Sample 1% of requests
export OTEL_SAMPLE_RATE=0.01

API Documentation Routes¶

stdapi.ai provides automatic API documentation routes, which are disabled by default for security in production environments.

ENABLE_DOCS¶

Purpose : Enable interactive Swagger UI documentation at /docs

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_DOCS=true

ENABLE_REDOC¶

Purpose : Enable ReDoc documentation UI at /redoc

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_REDOC=true

ENABLE_OPENAPI_JSON¶

Purpose : Enable OpenAPI schema JSON endpoint at /openapi.json

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_OPENAPI_JSON=true

Development Configuration¶

Enable all documentation routes for local development:

export ENABLE_DOCS=true
export ENABLE_REDOC=true
# ENABLE_OPENAPI_JSON is automatically enabled when ENABLE_DOCS or ENABLE_REDOC is true

Or enable only Swagger UI:

export ENABLE_DOCS=true
# ENABLE_OPENAPI_JSON is automatically enabled

Or enable only ReDoc:

export ENABLE_REDOC=true
# ENABLE_OPENAPI_JSON is automatically enabled

Production Best Practice¶

# Keep all routes disabled in production (default)
# No environment variables needed - defaults to false

Validation and Logging¶

For comprehensive logging and monitoring information, see the Logging and Monitoring guide.

STRICT_INPUT_VALIDATION¶

Purpose : Reject API requests containing unknown/extra fields instead of ignoring them

Type : Boolean

Default : false

# Returns HTTP 400 for requests with unexpected fields
export STRICT_INPUT_VALIDATION=true

CHAT_COMPLETIONS_REASONING_FIELD¶

Purpose : Choose which field carries a reasoning model's thinking text on /v1/chat/completions

Type : String

Default : reasoning_content

Options : reasoning_content, reasoning, none

Behavior : The OpenAI Chat Completions API returns no thinking text of its own — it reports only a reasoning_tokens count — so the providers that do return it have settled on two different names. reasoning_content is the DeepSeek spelling, which most clients that read reasoning at all look for first. reasoning is the name used by OpenRouter and vLLM. none emits neither, keeping responses strictly OpenAI-shaped. : The setting applies to both the completed message and the streamed deltas, so a client never sees one name while streaming and another at the end. Callers can also suppress reasoning per request with include_reasoning: false or reasoning: {"exclude": true}, whatever this is set to.

# Default: the name most clients read
export CHAT_COMPLETIONS_REASONING_FIELD=reasoning_content

# For clients written against OpenRouter or vLLM
export CHAT_COMPLETIONS_REASONING_FIELD=reasoning

# Strict OpenAI shape: never return thinking text
export CHAT_COMPLETIONS_REASONING_FIELD=none

LOG_LEVEL¶

Purpose : Control the minimum severity of log events written to STDOUT

Default : info

Options : info, warning, error, critical, disabled

Behavior : Only log events at or above the configured level are output. Log levels are ordered by severity: info < warning < error < critical

# Default: Output all log events
export LOG_LEVEL=info

# Production: Suppress info logs, show only warnings and higher
export LOG_LEVEL=warning

# Critical only: Show only critical errors
export LOG_LEVEL=critical

# Disable logging: Suppress all log output (not recommended)
export LOG_LEVEL=disabled

LOG_REQUEST_PARAMS¶

Purpose : Include request and response parameters (JSON body, form, query) in logs for integration debugging

Type : Boolean

Default : false

# Enable for debugging (NOT recommended for production)
export LOG_REQUEST_PARAMS=true

LOG_CLIENT_IP¶

Purpose : Enable logging of client IP addresses for each request and add IP to OpenTelemetry spans

Type : Boolean

Default : false (disabled for privacy)

# Disabled (default) - no client IP logging
# No environment variable needed

# Enable client IP logging
export LOG_CLIENT_IP=true

# Enable proxy headers to get real client IPs
export ENABLE_PROXY_HEADERS=true
# Enable client IP logging
export LOG_CLIENT_IP=true

# Only enable client IP logging
export LOG_CLIENT_IP=true
# ENABLE_PROXY_HEADERS remains false (default)

TIMEZONE¶

Purpose : IANA timezone identifier used for request date and time

Type : String (IANA timezone identifier)

Default : UTC

# UTC (default)
export TIMEZONE=UTC

# North America
export TIMEZONE=America/New_York

# Europe
export TIMEZONE=Europe/London

CloudWatch Metrics and Cost Tracking¶

The behavior of these settings — EMF line structure, cost log format, pricing accuracy, regional price fallback, known limitations, and the price override format with examples — is documented in CloudWatch Metrics (EMF) and Cost Tracking in the Logging and Monitoring guide.

CLOUDWATCH_METRICS¶

Purpose : Emit per-request AWS-billed usage as CloudWatch Embedded Metric Format (EMF) log lines

Type : Boolean

Default : false

export CLOUDWATCH_METRICS=true

CLOUDWATCH_METRICS_NAMESPACE¶

Purpose : CloudWatch namespace under which the emitted usage metrics are grouped

Type : String

Default : stdapi

Requirement : 1-255 characters, alphanumeric plus . - _ / # :, must not start with the reserved AWS/ prefix

export CLOUDWATCH_METRICS_NAMESPACE=my-app-metrics

COST_TRACKING¶

Purpose : Enable real-time cost computation from live AWS pricing (details and accuracy caveats). Disabled by default: it requires the extra pricing:GetProducts IAM permission — see Cost Tracking IAM Permissions.

Type : Boolean

Default : false

export COST_TRACKING=true

COST_PRICE_OVERRIDES¶

Purpose : Operator-supplied unit price overrides for models not covered by the AWS Price List API (format and example)

Type : JSON object — keys are model IDs, values are dicts mapping dimension name to price per one unit

Default : {}

Bedrock Guardrails¶

Amazon Bedrock Guardrails add content filtering and safety controls to model inputs and outputs. The configured guardrail also powers the OpenAI-compatible Moderations API (POST /v1/moderations); without one, that API falls back to inline guardrail checks in supported regions, then Amazon Comprehend.

Route Coverage¶

The configured guardrail applies to every route. Chat routes use the native Bedrock integration; routes whose AWS backend has no guardrail mechanism enforce it through the ApplyGuardrail API: client-supplied text is checked as INPUT before the backend call and generated text as OUTPUT after it.

Global Configuration¶

AWS_BEDROCK_GUARDRAIL_IDENTIFIER¶

Purpose : ID of the Bedrock Guardrail to apply

Required : Yes (together with AWS_BEDROCK_GUARDRAIL_VERSION)

export AWS_BEDROCK_GUARDRAIL_IDENTIFIER=abc123def456

AWS_BEDROCK_GUARDRAIL_VERSION¶

Purpose : Version of the Bedrock Guardrail

Required : Yes (together with AWS_BEDROCK_GUARDRAIL_IDENTIFIER)

export AWS_BEDROCK_GUARDRAIL_VERSION=1

AWS_BEDROCK_GUARDRAIL_TRACE¶

Purpose : Trace level for guardrail evaluation

Options : disabled, enabled, enabled_full

Default : None (optional)

export AWS_BEDROCK_GUARDRAIL_TRACE=enabled

AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE¶

Purpose : Control whether users can override the global guardrail configuration at request level via HTTP headers

Default : false (disabled for security)

Security Consideration : When set to false (default) and a global guardrail is configured, only the global configuration is enforced, preventing users from bypassing or modifying safety controls. Set to true if you need to allow per-request guardrail customization to override the global configuration.

Auto-Enable Behavior : If no global guardrail configuration is set (both AWS_BEDROCK_GUARDRAIL_IDENTIFIER and AWS_BEDROCK_GUARDRAIL_VERSION are unset), this setting is automatically set to true at startup, allowing per-request guardrails when no global policy is enforced.

export AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE=true

export AWS_BEDROCK_GUARDRAIL_IDENTIFIER=abc123def456
export AWS_BEDROCK_GUARDRAIL_VERSION=1
export AWS_BEDROCK_GUARDRAIL_TRACE=enabled
export AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE=false  # Default: prevent overrides

Per-Request Guardrail Configuration¶

Use HTTP headers to specify guardrail settings per request:

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-GuardrailIdentifier: abc123def456" \
  -H "X-Amzn-Bedrock-GuardrailVersion: 1" \
  -H "X-Amzn-Bedrock-Trace: enabled" \
  -d '{"model": "anthropic.claude-sonnet-5", "messages": [...]}'

Request Body Configuration¶

The amazon-bedrock-guardrailConfig object in the request body is supported for OpenAI Chat Completions compatibility.

Bedrock Session Storage¶

Requests with store=true on the Responses and Chat Completions APIs persist generations in Amazon Bedrock sessions. No environment variable is needed to enable this — it requires the Bedrock Session Storage IAM permissions.

AWS_BEDROCK_SESSION_ENCRYPTION_KEY_ARN¶

Purpose : KMS key ARN encrypting the Amazon Bedrock sessions that back stored responses and stored chat completions (store=true)

Default : None — sessions are encrypted with the AWS-managed key

Validation : Checked at startup: must be a KMS key ARN (arn:<partition>:kms:<region>:<account-id>:key/<key-id>).

export AWS_BEDROCK_SESSION_ENCRYPTION_KEY_ARN=arn:aws:kms:us-east-1:123456789012:key/abcd-...

Bedrock Service Tier and Performance Configuration¶

Amazon Bedrock service tiers and performance configurations allow you to optimize AI workload performance and cost trade-offs. Configure latency optimization and throughput priority for your inference requests.

Service Tiers¶

Service tiers help you match AI workload performance with cost by selecting the appropriate throughput and latency characteristics:

• priority - Highest priority processing with guaranteed capacity and fastest response times. Best for latency-sensitive applications.
• default - Standard processing with balanced performance and cost. Suitable for most production workloads.
• flex - Cost-optimized processing with flexible scheduling. Best for batch jobs and non-time-sensitive workloads.

Performance Configuration¶

Performance configuration allows you to optimize for latency:

• standard - Standard latency profile with balanced performance
• optimized - Optimized for lowest possible latency

Per-Request Service Tier Configuration¶

Configure service tier and performance settings per request using HTTP headers. These headers are available on all Bedrock-based routes (Chat Completions, Embeddings, Images). Server-side per-model defaults can be set with DEFAULT_MODEL_SERVICE_TIERS.

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: priority" \
  -H "X-Amzn-Bedrock-PerformanceConfig-Latency: optimized" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-5",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

curl -X POST https://api.example.com/v1/embeddings \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: flex" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-2-multimodal-embeddings-v1:0",
    "input": ["text 1", "text 2", "text 3"]
  }'

curl -X POST https://api.example.com/v1/images/generations \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: default" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-canvas-v1:0",
    "prompt": "A serene mountain landscape"
  }'

Audio and Text-to-Speech¶

DEFAULT_TTS_MODEL¶

Purpose : Default text-to-speech model when not specified in requests

Default : amazon.polly-standard

export DEFAULT_TTS_MODEL=amazon.polly-neural

DEFAULT_TTS_LANGUAGE¶

Purpose : Default language code for text-to-speech synthesis when using OpenAI voice names

Default : None (automatic language detection via Amazon Comprehend)

Behavior : When specified, this language is used instead of automatic detection. When not set, Amazon Comprehend detects the language automatically from the input text.

Valid Language Codes: Any Amazon Polly language code (e.g., en-US, fr-FR, es-ES, de-DE, ja-JP)

# Use English (US) for all TTS requests
export DEFAULT_TTS_LANGUAGE=en-US

# Use French for all TTS requests
export DEFAULT_TTS_LANGUAGE=fr-FR

Deprecated Settings¶

Model Cache¶

stdapi.ai automatically discovers and caches available Bedrock models from configured regions. The cache is refreshed on-demand when expired, not via background tasks.

MODEL_CACHE_SECONDS¶

Purpose : Cache lifetime for the Bedrock models list before refresh

Type : Integer (seconds)

Default : 900 (15 minutes)

Behavior : When a request needs the model list (e.g., model lookup, /models endpoint) and the cache has expired, the server queries Amazon Bedrock to discover newly available models, check for model access changes, and update inference profile configurations. This cache also applies to application inference profile and prompt router information when users pass ARNs directly (if enabled via AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN or AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN)

# Default: 15 minutes
export MODEL_CACHE_SECONDS=900

# More frequent updates (5 minutes)
export MODEL_CACHE_SECONDS=300

# Less frequent updates (1 hour)
export MODEL_CACHE_SECONDS=3600

AI Response Timeout¶

AI_RESPONSE_TIMEOUT¶

Purpose : Maximum time in seconds to wait without receiving any data from an AI model

Type : Integer (seconds, must be greater than 0)

Default : 600 (10 minutes)

Behavior : Inactivity (per-read) timeout on the upstream model connection, applied to both streaming and non-streaming requests. The timer resets every time data is received, so it fires only when the model stalls for longer than this value — it does not bound the total duration of a response: a stream that keeps producing chunks can run well past it. On a non-streaming request, where the whole response arrives at once, it effectively bounds the wait for that single response. When it fires, the connection is closed and the request fails with a timeout error

# Default (10 minutes) - suitable for extended thinking models
export AI_RESPONSE_TIMEOUT=600

# Shorter timeout for standard models (2 minutes)
export AI_RESPONSE_TIMEOUT=120

# Longer timeout for very long documents or high reasoning budgets (15 minutes)
export AI_RESPONSE_TIMEOUT=900

Default Model Parameters¶

Configure default inference parameters applied automatically to specific models.

DEFAULT_MODEL_PARAMS¶

Purpose : Per-model default parameters

Format : JSON object with model IDs as keys

Supported Parameters:

Only the outer JSON shape (an object of per-model objects) is validated at startup. The parameter values above are validated lazily, the first time a model with configured defaults is used: a wrong type, or a value below the lower bounds shown in the table, fails that request with HTTP 400. The numeric ceilings (for example the usual top_p maximum of 1.0) are enforced by Amazon Bedrock and the target model.

Configuration Examples¶

Basic Parameters:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-micro-v1:0": {
    "temperature": 0.3,
    "max_tokens": 800
  }
}'

Provider-Specific Features:

export DEFAULT_MODEL_PARAMS='{
  "anthropic.claude-sonnet-5": {
    "anthropic_beta": ["Interleaved-thinking-2025-05-14"]
  }
}'

Multiple Models:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-micro-v1:0": {
    "temperature": 0.3,
    "max_tokens": 500
  },
  "amazon.nova-lite-v1:0": {
    "temperature": 0.7,
    "max_tokens": 2000
  },
  "anthropic.claude-sonnet-5": {
    "temperature": 0.5,
    "top_p": 0.9,
    "anthropic_beta": ["Interleaved-thinking-2025-05-14"]
  }
}'

Advanced Configuration:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-pro-v1:0": {
    "temperature": 0.7,
    "top_p": 0.95,
    "max_tokens": 4096,
    "stop_sequences": ["Human:", "Assistant:"]
  }
}'

Parameter Merging¶

graph LR
    A[Default Parameters] --> B[Merged Config]
    C[Request Parameters] --> B
    B --> D[Final Configuration]

1. Default parameters are applied first (from DEFAULT_MODEL_PARAMS)
2. Request parameters override defaults if both are specified
3. Provider-specific fields are forwarded to Bedrock as additional model request fields
4. Unsupported fields reach Bedrock as-is, and a field the model rejects surfaces as a ValidationException returned to the client as HTTP 400. Three cases are handled before that: anthropic_beta flags are filtered individually against an allowlist (see ANTHROPIC_BETA_FILTER); a system prompt sent to a model that does not support one is dropped when DROP_UNSUPPORTED_SYSTEM_PROMPT is enabled (the default); and Amazon Nova 2 drops max_tokens when reasoning effort is high, logging a warning

Default Model Service Tiers¶

Configure default service tiers applied automatically to specific Bedrock models.

DEFAULT_MODEL_SERVICE_TIERS¶

Purpose : Per-model default service tier

Format : JSON object with model IDs as keys and tier string as value

Default : {}

Supported Values:

Configuration Examples¶

Single Model:

export DEFAULT_MODEL_SERVICE_TIERS='{
  "amazon.nova-pro-v1:0": "flex"
}'

Multiple Models:

export DEFAULT_MODEL_SERVICE_TIERS='{
  "amazon.nova-pro-v1:0": "flex",
  "amazon.nova-premier-v1:0": "priority"
}'

Service Tier Merging¶