Configuration Guide¶

stdapi.ai is configured entirely through environment variables, which are read once at startup and cannot be changed without restarting the service. This guide explains each setting category with practical examples to help you configure the service correctly.

What you can configure:

• AWS regions - Access models across multiple regions for availability and model selection
• Data sovereignty - Control which AWS regions are used for compliance (GDPR, HIPAA, etc.)
• Storage - S3 buckets for file operations, regional buckets for multi-region deployments
• Authentication - API keys via SSM or Secrets Manager for secure access control
• Observability - Logging levels, OpenTelemetry, request/response debugging
• Security - CORS, proxy headers, trusted hosts for production deployments
• Performance - Caching, model overrides, S3 acceleration
• TLS / SSL - End-to-end encryption using Granian environment variables

Quick Start¶

For production deployments, configure these essential settings:

Minimal Production Setup¶

Single-region deployment with file storage only.

# S3 bucket for file storage (must be in same region as your server)
export AWS_S3_BUCKET=my-stdapi-bucket

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Production with Authentication¶

Adds secure API key authentication via AWS Systems Manager.

# S3 bucket for file storage (must be in same region as your server)
export AWS_S3_BUCKET=my-stdapi-bucket

# Secure API authentication (recommended: SSM Parameter Store)
export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Full Production Setup (All Features Enabled)¶

Multi-region deployment with all AWS AI services, observability, and security features.

# Core AWS configuration - host server in first region
export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,eu-west-1

# S3 bucket for file storage (must be in us-east-1, your first/primary region)
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket

# Optional: Transcribe S3 bucket (defaults to AWS_S3_BUCKET if not specified)
# Only set this if you need a separate bucket or if transcribe is in a different region
# export AWS_TRANSCRIBE_S3_BUCKET=my-stdapi-transcribe-us-east-1

# Optional: Regional buckets for async/batch inference in other regions
export AWS_S3_REGIONAL_BUCKETS='{"us-west-2": "my-stdapi-us-west-2-bucket", "eu-west-1": "my-stdapi-eu-west-1-bucket"}'

# AWS AI services regions (optional - defaults to first AWS_BEDROCK_REGIONS if not specified)
export AWS_POLLY_REGION=us-east-1           # Text-to-speech
export AWS_TRANSCRIBE_REGION=us-east-1      # Speech-to-text (audio transcription)
export AWS_COMPREHEND_REGION=us-east-1      # Language detection
export AWS_TRANSLATE_REGION=us-east-1       # Text translation

# Authentication
export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

# Logging
export LOG_LEVEL=warning
export LOG_CLIENT_IP=true

# Optional: OpenTelemetry observability (AWS X-Ray integration)
# export OTEL_ENABLED=true
# export OTEL_SERVICE_NAME=stdapi-production
# export OTEL_SAMPLE_RATE=0.1

# Production security settings (when behind AWS ALB/CloudFront)
export ENABLE_PROXY_HEADERS=true

# Note: TRUSTED_HOSTS not recommended with AWS ALB - use ALB host-based routing instead
# Only use TRUSTED_HOSTS if you cannot configure host validation at the load balancer level

# Optional: CORS for browser-based web applications
# export CORS_ALLOW_ORIGINS='["https://app.example.com"]'

Development Setup¶

Local development configuration with API documentation and debug logging enabled.

# Minimal configuration for local development
export AWS_S3_BUCKET=my-stdapi-dev-bucket

# Enable API documentation
export ENABLE_DOCS=true
export ENABLE_REDOC=true

# Full request/response logging for debugging
export LOG_LEVEL=info
export LOG_REQUEST_PARAMS=true

# AWS_BEDROCK_REGIONS is optional - will auto-detect your current AWS region if not specified

Environment Variable Summary¶

This section provides a quick reference of all available configuration options. Detailed explanations for each variable can be found in the sections below.

Essential (Production)¶

AWS Client¶

AWS Storage¶

AWS AI Services¶

Resilience & Failover¶

Bedrock Advanced¶

Authentication¶

Choose one method (mutually exclusive):

API Compatibility¶

Logging¶

Observability (OpenTelemetry)¶

HTTP/Security¶

Application Behavior¶

API Documentation¶

AWS Services and Regions¶

General Configuration¶

AWS_ADAPTIVE_RETRY¶

Purpose : Enable adaptive retry mode that adjusts retry pacing based on observed error rates across all AWS service calls

Type : Boolean (true / false)

Default : false

Behavior : When enabled, the retry strategy dynamically responds to real-time congestion signals. If errors are occurring frequently, retries are spaced further apart to avoid amplifying load on an already-stressed endpoint. Once conditions improve, the pacing returns to normal. When disabled, retries follow a standard exponential backoff strategy with fixed intervals. Applies to all AWS services (Bedrock, S3, Polly, Transcribe, etc.).

# Default: standard exponential backoff
export AWS_ADAPTIVE_RETRY=false

# Enable adaptive retry (recommended under sustained high load)
export AWS_ADAPTIVE_RETRY=true

AWS_MAX_POOL_CONNECTIONS¶

Purpose : Maximum number of concurrent HTTP connections per AWS service client

Type : Integer (must be > 0)

Default : 50

Behavior : Each AWS service client (one per service per region) maintains its own connection pool up to this limit. Under high concurrency, increasing this value prevents requests from queuing for an available connection. Setting it too high may exhaust system file descriptors.

# Default
export AWS_MAX_POOL_CONNECTIONS=50

# High-concurrency deployment
export AWS_MAX_POOL_CONNECTIONS=100

AWS_CONNECT_TIMEOUT¶

Purpose : Timeout in seconds for establishing a connection to an AWS service endpoint

Type : Integer (must be > 0)

Default : 5

Behavior : Limits how long the client waits when opening a new connection. A short value allows fast failover to another region when an endpoint is unreachable. Increase it only if you see spurious connection timeouts on high-latency networks.

# Default: 5 seconds
export AWS_CONNECT_TIMEOUT=5

# High-latency network
export AWS_CONNECT_TIMEOUT=10

Storage Configuration¶

AWS_S3_BUCKET¶

Purpose : Primary S3 bucket for storing generated files (images, audio, documents) and temporary data during processing

Default : None (must be configured for file operations)

Best Practice : The bucket must be in the first region specified in AWS_BEDROCK_REGIONS (your primary region where the server should be hosted) to avoid cross-region data transfer costs and reduce latency

export AWS_S3_BUCKET=my-llm-storage-us-east-1

AWS_S3_ACCELERATE¶

Purpose : Enable S3 Transfer Acceleration for presigned URLs to improve download performance for large files

Type : Boolean

Default : false

Best Practice : Enable when serving large files (high-resolution images, audio) to geographically distributed users

export AWS_S3_ACCELERATE=true

AWS_S3_TMP_PREFIX¶

Purpose : S3 prefix (folder path) for temporary files used during job processing

Default : tmp/

Best Practice : Configure S3 lifecycle policies to automatically delete objects under this prefix after 1 day

export AWS_S3_TMP_PREFIX=tmp/

Custom prefix examples:

# Production environment
export AWS_S3_TMP_PREFIX=prod/tmp/

# Staging environment
export AWS_S3_TMP_PREFIX=staging/tmp/

# Organize by date (requires manual updates)
export AWS_S3_TMP_PREFIX=tmp/2025/01/

# No prefix (store at bucket root - not recommended)
export AWS_S3_TMP_PREFIX=

AWS_S3_FILES_PREFIX¶

Purpose : S3 prefix (folder path) for Files API objects (OpenAI and Anthropic /v1/files endpoints)

Default : files/

Best Practice : Configure an AbortIncompleteMultipartUpload S3 lifecycle rule on this prefix to clean up abandoned upload parts, and apply Intelligent-Tiering for cost optimisation

export AWS_S3_FILES_PREFIX=files/

Custom prefix examples:

# Production environment
export AWS_S3_FILES_PREFIX=prod/files/

# Staging environment
export AWS_S3_FILES_PREFIX=staging/files/

# No prefix (store at bucket root - not recommended)
export AWS_S3_FILES_PREFIX=

AWS_TRANSCRIBE_S3_BUCKET¶

Purpose : Temporary S3 bucket for transcription workflows

Default : Falls back to AWS_S3_BUCKET if not specified

Requirement : Must be in the same region as AWS_TRANSCRIBE_REGION

# If AWS_TRANSCRIBE_REGION is us-east-1
export AWS_TRANSCRIBE_S3_BUCKET=my-transcribe-temp-us-east-1

# If AWS_TRANSCRIBE_REGION is eu-west-1
export AWS_TRANSCRIBE_S3_BUCKET=my-transcribe-temp-eu-west-1

AWS_S3_REGIONAL_BUCKETS¶

Purpose : Region-specific S3 buckets for Bedrock async and batch inference operations

Default : Empty (no regional buckets configured)

Format : JSON object with region names as keys and bucket names as values

Requirement : Some Bedrock models require S3 buckets in the same region for async and batch inference operations

export AWS_S3_REGIONAL_BUCKETS='{"us-east-1": "my-bedrock-temp-us-east-1", "eu-west-1": "my-bedrock-temp-eu-west-1"}'

AWS_S3_ACCEPTED_BUCKETS¶

Purpose : Declare external S3 buckets that the application has read access to, mapped to their AWS region

Type : JSON object (keys: bucket names, values: AWS region identifiers)

Default : {} (empty — only the application's own buckets are recognized)

Behavior : Buckets listed here are recognized for two purposes:

- **S3 HTTP URL to S3 URI conversion** — When a user passes an S3 HTTP URL (including presigned URLs) for one of these buckets, it is automatically converted to an `s3://` URI so Bedrock can access the object directly.
- **Region-aware routing** — The router knows the region of these buckets and can factor it into region selection to minimize cross-region data transfer.

Without this setting, only the application's own buckets (`AWS_S3_BUCKET` and `AWS_S3_REGIONAL_BUCKETS`) are recognized.

export AWS_S3_ACCEPTED_BUCKETS='{"my-data-bucket": "us-east-1", "my-eu-bucket": "eu-west-1"}'

{
  "Effect": "Allow",
  "Action": "s3:GetObject",
  "Resource": [
    "arn:aws:s3:::my-data-bucket/*",
    "arn:aws:s3:::my-eu-bucket/*"
  ]
}

S3 Bucket Lifecycle Configuration¶

Purpose : Configure automatic deletion of temporary files and abandoned multipart upload parts to minimize storage costs

Recommendation : Configure S3 lifecycle policies to automatically delete objects under the AWS_S3_TMP_PREFIX after 1 day, and abort incomplete multipart uploads under the AWS_S3_FILES_PREFIX after 1 day

stdapi.ai stores temporary files under the prefix configured by AWS_S3_TMP_PREFIX (default: tmp/). These include generated images, audio files, and transcription workflow files. Configure S3 lifecycle policies to automatically delete objects under this prefix after 1 day.

Additionally, multipart file uploads (OpenAI Uploads API) store parts under AWS_S3_FILES_PREFIX (default: files/). If a session is never completed or cancelled — for example when a client disconnects — the uploaded parts remain in S3 and accumulate costs. Add an AbortIncompleteMultipartUpload rule on the files prefix to clean these up automatically.

{
  "Rules": [
    {
      "Id": "DeleteTemporaryFiles",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "tmp/"
      },
      "Expiration": {
        "Days": 1
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 1
      }
    },
    {
      "Id": "AbortIncompleteMultipartUploads",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "files/"
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 1
      }
    }
  ]
}

Apply via AWS CLI:

# For primary S3 bucket (AWS_S3_BUCKET)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-stdapi-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

# For transcribe S3 bucket (AWS_TRANSCRIBE_S3_BUCKET, if different from AWS_S3_BUCKET)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-transcribe-temp-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

# For regional buckets (AWS_S3_REGIONAL_BUCKETS)
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-stdapi-us-west-2-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

Bedrock Configuration¶

AWS_BEDROCK_REGIONS¶

Purpose : List of AWS regions where Bedrock models are available

Format : Comma-separated string

Default : Current AWS SDK region if not specified

Behavior : Models are discovered in the same order as the listed regions. The first region is the primary region where your server should be hosted on AWS for optimal performance. Your S3 bucket (aws_s3_bucket) must also be in this region. If a model is unavailable in the primary region, subsequent regions are checked in order

export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,eu-west-1

AWS_BEDROCK_CROSS_REGION_INFERENCE¶

Purpose : Enable automatic cross-region routing when a model isn't available in the primary region

Type : Boolean

Default : true

export AWS_BEDROCK_CROSS_REGION_INFERENCE=true

AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL¶

Purpose : Allow global cross-region inference routing to any region worldwide

Type : Boolean

Default : true

export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

AWS_BEDROCK_REGION_ROUTING¶

Purpose : Automatic region routing strategy for distributing Bedrock requests across configured regions

Type : String

Default : ordered

Behavior : When multiple regions are configured in AWS_BEDROCK_REGIONS, this setting controls how requests are distributed across them. The router automatically handles quota/throttling errors and regional unavailability by temporarily avoiding affected regions

Requirement : Requires at least 2 regions in AWS_BEDROCK_REGIONS to take effect

Available strategies:

# Use ordered routing (default)
export AWS_BEDROCK_REGION_ROUTING=ordered

# Use lowest latency routing
export AWS_BEDROCK_REGION_ROUTING=lowest_latency

# Disable routing
export AWS_BEDROCK_REGION_ROUTING=disabled

AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS¶

Purpose : Duration to temporarily avoid a region after receiving a quota or throttling error

Type : Integer (seconds, must be > 0)

Default : 60

Behavior : This is the base backoff value. When a Bedrock API call fails due to quota limits (ThrottlingException, TooManyRequestsException, ServiceQuotaExceededException), the affected region is temporarily blocked. The actual delay doubles with each consecutive quota error on the same region (exponential backoff), up to a hard ceiling of 1 hour. The counter resets after a successful request. Subsequent requests are routed to other available regions during the backoff period.

# Default: 60 seconds (base value — actual delay doubles per consecutive error)
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=60

# Shorter base backoff for aggressive retry
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=30

# Longer base backoff for conservative approach
export AWS_BEDROCK_REGION_ROUTING_QUOTA_BACKOFF_SECONDS=120

AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS¶

Purpose : Duration to temporarily avoid a region after receiving an unavailability error

Type : Integer (seconds, must be > 0)

Default : 30

Behavior : When a Bedrock API call fails due to service unavailability (ServiceUnavailableException, ModelNotReadyException), the affected region is temporarily blocked for this many seconds. These errors are typically shorter-lived than quota limits, so the default is shorter

# Default: 30 seconds
export AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS=30

# Longer backoff for stability
export AWS_BEDROCK_REGION_ROUTING_UNAVAILABLE_BACKOFF_SECONDS=60

AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS¶

Purpose : Hard ceiling in seconds on the exponential quota backoff for a single region

Type : Integer (seconds, must be > 0)

Default : 3600 (1 hour)

Behavior : Quota backoff grows exponentially with consecutive errors (base interval × 2^n). This setting caps how large that value can become, preventing a region from being blocked indefinitely. Reduce it to allow faster recovery; increase it to keep a misbehaving region sidelined for longer.

# Default: 1 hour ceiling
export AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS=3600

# More aggressive recovery
export AWS_BEDROCK_REGION_ROUTING_MAX_QUOTA_BACKOFF_SECONDS=600

AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR¶

Purpose : Multiplier applied to the max quota backoff to compute the stale-error reset threshold

Type : Integer (must be > 0)

Default : 2 (threshold = 2 × max quota backoff = 2 hours with defaults)

Behavior : If the most recent quota error on a region occurred more than max_quota_backoff × factor seconds ago, the consecutive-error counter is reset and the next error is treated as a fresh start rather than an escalation. A higher value keeps memory of past errors for longer before resetting the counter.

# Default: reset counter after 2× the max backoff window
export AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR=2

# Longer memory of past errors
export AWS_BEDROCK_REGION_ROUTING_QUOTA_STALE_FACTOR=4

AWS_BEDROCK_MAX_RETRIES¶

Purpose : Total number of retries per Bedrock invocation, cycling through available regions in order

Type : Integer (must be > 0)

Default : 9

Behavior : Controls the total retry budget for each Bedrock API call. When region routing is enabled, retries cycle through the available regions in priority order — after exhausting all regions the cycle starts again. For example, with 3 regions and 9 retries the attempt sequence is r1, r2, r3, r1, r2, r3, r1, r2, r3, r1 (1 initial + 9 retries = 10 total attempts). When routing is disabled, retries are performed against the single configured region.

# Default: 9 retries (10 total attempts)
export AWS_BEDROCK_MAX_RETRIES=9

# Fail faster (e.g. low-latency interactive use cases)
export AWS_BEDROCK_MAX_RETRIES=3

# More resilient (e.g. batch workloads tolerant of longer waits)
export AWS_BEDROCK_MAX_RETRIES=18

AWS_BEDROCK_MODEL_REGION_RESTRICT¶

Purpose : Restrict a model to specific region(s) only, useful when a model provides important features only in certain regions

Type : JSON object (keys: Bedrock model IDs or prefixes, values: ordered lists of allowed regions)

Default : {} (empty — no model-specific region restriction)

Behavior : When set, the model is made available only in the listed regions. No fallback to other regions occurs. The order of the list determines routing priority when multiple regions are listed. Keys can be exact model IDs or prefixes that match the beginning of a model ID

# Restrict Nova Pro to us-east-1 for grounding support
export AWS_BEDROCK_MODEL_REGION_RESTRICT='{"amazon.nova-pro-v1:0": ["us-east-1"]}'

AWS_BEDROCK_LEGACY¶

Purpose : Allow usage of legacy/deprecated Bedrock models

Type : Boolean

Default : false

export AWS_BEDROCK_LEGACY=true

AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK¶

Purpose : Transparently reroute requests using a deprecated model ID to its recommended replacement

Type : Boolean

Default : true

Behavior : When true, any request that specifies a deprecated model ID (as listed in the server's deprecation registry) is silently retried with the recommended replacement model. The replacement is fully re-evaluated — alias resolution, modality checks, and region routing all apply to the new model ID. When false, deprecated model IDs return a 404 error with a message indicating the replacement, forcing clients to migrate explicitly.

# Transparent fallback (default) — clients using old model IDs keep working
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=true

# Strict mode — deprecated model IDs return 404, clients must update their code
export AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK=false

AWS_BEDROCK_DEPRECATED_MODELS¶

Purpose : Extend or override the built-in deprecated model registry with custom mappings

Type : JSON object — dict[str, str]

Default : {}

Behavior : Merged with the built-in registry at startup. User-provided entries take precedence over built-in ones — this means it can be used both to add new deprecated model mappings and to override the fallback target of an already-defined deprecated model. Effective only when AWS_BEDROCK_DEPRECATED_MODEL_FALLBACK is true.

Reference : AWS Bedrock model lifecycle

# Add a custom deprecated model and override an existing built-in mapping
export AWS_BEDROCK_DEPRECATED_MODELS='{"my-old-model-v1": "my-new-model-v2", "amazon.titan-text-lite-v1": "amazon.nova-lite-v1:0"}'

AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE¶

Purpose : Control automatic subscription to new models in AWS Marketplace

Type : Boolean

Default : true

Behavior : When true, the server automatically subscribes to new models discovered in the AWS Marketplace, making them immediately available through the API. When false, only models with existing marketplace subscriptions are visible and accessible

IAM Permissions Required : aws-marketplace:Subscribe, aws-marketplace:ViewSubscriptions

# Allow automatic subscription (default)
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=true

# Restrict to pre-subscribed models only
export AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=false

AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass cross-region inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use cross-region inference profile ARNs instead of model IDs in the model parameter. Cross-region inference profiles enable routing to multiple regions for better availability

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable cross-region inference profile ARN support
export AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0

AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN¶

Purpose : Allow users to pass application inference profile ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use application inference profile ARNs instead of model IDs in the model parameter. Application inference profiles are custom routing configurations for specific use cases

IAM Permissions Required : bedrock:GetInferenceProfile (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable application inference profile ARN support
export AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN=true

arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123xyz

AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN¶

Purpose : Allow users to pass prompt router ARNs directly as model IDs in API requests

Type : Boolean

Default : false

Behavior : When enabled, users can use prompt router ARNs instead of model IDs in the model parameter. Prompt routers enable dynamic model selection based on prompt characteristics

IAM Permissions Required : bedrock:GetPromptRouter (see IAM Permissions)

# Disabled (default) - users can only use standard model IDs
# No environment variable needed

# Enable prompt router ARN support
export AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN=true

arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router

AWS_BEDROCK_MODEL_ARN_MAPPING¶

Purpose : Map standard model IDs to custom inference profile or prompt router ARNs for server-controlled routing

Format : JSON object with model IDs as keys and ARNs as values

Default : {} (empty, no mappings)

Behavior : When configured, the mapped ARN is used instead of the default cross-region inference profile when clients request the model by its standard ID. This provides centralized control over model routing without requiring client changes

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-3-5-sonnet-20241022-v2:0": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-custom-profile",
  "anthropic.claude-3-5-haiku-20241022-v1:0": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/my-router"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-3-5-sonnet-20241022-v2:0": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/cost-optimizer"
}'

export AWS_BEDROCK_MODEL_ARN_MAPPING='{
  "anthropic.claude-3-5-sonnet-20241022-v2:0": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/production-profile"
}'

# Production: Use cost-optimized prompt router
export AWS_BEDROCK_MODEL_ARN_MAPPING='{"anthropic.claude-3-5-sonnet-20241022-v2:0": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/prod-router"}'

# Development: Use standard cross-region profile
export AWS_BEDROCK_MODEL_ARN_MAPPING='{}'

Other AWS Services¶

AWS_POLLY_REGION¶

Purpose : Region for Amazon Polly text-to-speech service

Default : First region in AWS_BEDROCK_REGIONS

export AWS_POLLY_REGION=us-east-1

AWS_COMPREHEND_REGION¶

Purpose : Region for Amazon Comprehend language detection service

Default : First region in AWS_BEDROCK_REGIONS

export AWS_COMPREHEND_REGION=us-east-1

AWS_TRANSCRIBE_REGION¶

Purpose : Region for Amazon Transcribe speech-to-text service

Default : First region in AWS_BEDROCK_REGIONS

export AWS_TRANSCRIBE_REGION=us-east-1

AWS_TRANSLATE_REGION¶

Purpose : Region for Amazon Translate text translation service

Default : First region in AWS_BEDROCK_REGIONS

export AWS_TRANSLATE_REGION=us-east-1

Compliance and Latency Optimization¶

Strategic region configuration is critical for both regulatory compliance and performance optimization. This section provides best practice configurations for common scenarios.

GDPR and Data Residency Compliance¶

For applications serving European users, data residency regulations like GDPR may require that data processing occurs within specific geographic boundaries.

# Use only European regions
export AWS_S3_BUCKET=my-stdapi-eu-bucket
export AWS_BEDROCK_REGIONS=eu-west-1,eu-west-3,eu-central-1

# Disable global cross-region inference to prevent data routing outside Europe
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

# Keep cross-region inference enabled for failover within EU regions
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true

Latency Optimization¶

For applications prioritizing low latency and high performance, configure regions closest to your users and application infrastructure.

North America:

# Primary region for lowest latency, with fallbacks
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket
export AWS_BEDROCK_REGIONS=us-east-1,us-west-2,us-east-2

# Enable all cross-region inference for maximum model availability
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Asia-Pacific:

# Use Asia-Pacific regions for lowest latency to APAC users
export AWS_S3_BUCKET=my-stdapi-ap-southeast-1-bucket
export AWS_BEDROCK_REGIONS=ap-southeast-1,ap-northeast-1,us-west-2

# Enable global inference for fallback to US regions if needed
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Global Multi-Region:

# Balanced configuration with worldwide coverage
export AWS_S3_BUCKET=my-stdapi-us-east-1-bucket
export AWS_BEDROCK_REGIONS=us-east-1,eu-west-1,ap-southeast-1,us-west-2

# Enable global inference for best availability
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=true

Hybrid Approach: Compliance with Performance¶

Balance compliance requirements with performance needs:

# EU primary with US fallback (for model availability)
export AWS_S3_BUCKET=my-stdapi-eu-bucket
export AWS_BEDROCK_REGIONS=eu-west-1,eu-central-1,us-east-1

# Allow cross-region but restrict to specific regions only
export AWS_BEDROCK_CROSS_REGION_INFERENCE=true
export AWS_BEDROCK_CROSS_REGION_INFERENCE_GLOBAL=false

Configuration Order¶

When deploying stdapi.ai, configure settings in this recommended order:

1. IAM Permissions - Set up AWS access first
2. AWS Services and Regions - Configure S3 buckets and Bedrock regions
3. Authentication - Secure your API with authentication
4. Optional Features - Add observability, guardrails, and other features as needed

IAM Permissions¶

stdapi.ai requires specific AWS IAM permissions to access Bedrock models and other AWS services. The exact permissions needed depend on which features you enable.

Bedrock (Required)¶

Environment Variables: Always required

These permissions are mandatory for stdapi.ai to discover and invoke Bedrock models:

{
  "Sid": "BedrockModelInvoke",
  "Effect": "Allow",
  "Action": [
    "bedrock:CountTokens",
    "bedrock:GetAsyncInvoke",
    "bedrock:InvokeModel",
    "bedrock:InvokeModelWithResponseStream",
    "bedrock:InvokeTool"
  ],
  "Resource": "*"
},
{
  "Sid": "BedrockAsyncInvokeTagging",
  "Effect": "Allow",
  "Action": [
    "bedrock:TagResource"
  ],
  "Resource": "arn:aws:bedrock:*:*:async-invoke/*"
},
{
  "Sid": "BedrockModelDiscovery",
  "Effect": "Allow",
  "Action": [
    "bedrock:ListFoundationModels",
    "bedrock:GetFoundationModelAvailability",
    "bedrock:ListProvisionedModelThroughputs",
    "bedrock:ListInferenceProfiles"
  ],
  "Resource": "*"
}

Bedrock Marketplace Auto-Subscribe (Optional)¶

Environment Variables: AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE

Required only if you want to enable automatic subscription to new models in the AWS Marketplace (AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE=true, which is the default). When enabled, the server can automatically subscribe to marketplace offerings for newly discovered models.

{
  "Sid": "BedrockMarketplaceAutoSubscribe",
  "Effect": "Allow",
  "Action": [
    "aws-marketplace:Subscribe",
    "aws-marketplace:ViewSubscriptions"
  ],
  "Resource": "*"
}

Bedrock Inference Profiles and Prompt Routers (Optional)¶

Environment Variables: AWS_BEDROCK_ALLOW_CROSS_REGION_INFERENCE_PROFILE_ARN, AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN, AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN, AWS_BEDROCK_MODEL_ARN_MAPPING

Required only if you enable ARN-based routing features that allow users to pass inference profile or prompt router ARNs directly as model IDs, or if you configure server-side ARN mappings.

{
  "Sid": "BedrockInferenceProfilesAndPromptRouters",
  "Effect": "Allow",
  "Action": [
    "bedrock:GetInferenceProfile",
    "bedrock:GetPromptRouter"
  ],
  "Resource": "*"
}

Bedrock Guardrails (Optional)¶

Environment Variables: AWS_BEDROCK_GUARDRAIL_IDENTIFIER, AWS_BEDROCK_GUARDRAIL_VERSION

Required only if you configure Bedrock Guardrails for content filtering. See Bedrock Guardrails configuration section.

{
  "Sid": "BedrockGuardrails",
  "Effect": "Allow",
  "Action": [
    "bedrock:ApplyGuardrail"
  ],
  "Resource": "arn:aws:bedrock:*:*:guardrail/*"
}

S3 File Storage (Optional)¶

Environment Variables: AWS_S3_BUCKET

Required for storing generated images, audio files, and documents. See Storage Configuration for bucket setup details.

{
  "Sid": "S3FileStorage",
  "Effect": "Allow",
  "Action": [
    "s3:PutObject",
    "s3:PutObjectTagging",
    "s3:GetObject",
    "s3:DeleteObject",
    "s3:CreateMultipartUpload",
    "s3:UploadPart",
    "s3:CompleteMultipartUpload",
    "s3:AbortMultipartUpload",
    "s3:ListMultipartUploadParts"
  ],
  "Resource": "arn:aws:s3:::AWS_S3_BUCKET_VALUE/*"
},
{
  "Sid": "S3FileStorageList",
  "Effect": "Allow",
  "Action": [
    "s3:ListBucket",
    "s3:ListBucketMultipartUploads"
  ],
  "Resource": "arn:aws:s3:::AWS_S3_BUCKET_VALUE"
}

{
  "Sid": "KMSEncryptedBucket",
  "Effect": "Allow",
  "Action": [
    "kms:Decrypt",
    "kms:GenerateDataKey"
  ],
  "Resource": "arn:aws:kms:REGION:ACCOUNT_ID:key/YOUR_KMS_KEY_ID",
  "Condition": {
    "StringEquals": {
      "kms:ViaService": "s3.REGION.amazonaws.com"
    }
  }
}

Text-to-Speech (Optional)¶

Environment Variables: AWS_POLLY_REGION, DEFAULT_TTS_MODEL, DEFAULT_TTS_LANGUAGE

Required for generating speech from text using Amazon Polly. See Audio and Text-to-Speech configuration section.

{
  "Sid": "PollyTextToSpeech",
  "Effect": "Allow",
  "Action": [
    "polly:SynthesizeSpeech",
    "polly:DescribeVoices"
  ],
  "Resource": "*"
}

Speech-to-Text (Optional)¶

Environment Variables: AWS_TRANSCRIBE_REGION, AWS_TRANSCRIBE_S3_BUCKET

Required for transcribing audio files using Amazon Transcribe.

{
  "Sid": "TranscribeSpeechToText",
  "Effect": "Allow",
  "Action": [
    "transcribe:StartTranscriptionJob",
    "transcribe:GetTranscriptionJob",
    "transcribe:DeleteTranscriptionJob"
  ],
  "Resource": "*"
},
{
  "Sid": "TranscribeTagging",
  "Effect": "Allow",
  "Action": [
    "transcribe:TagResource"
  ],
  "Resource": "arn:aws:transcribe:*:*:transcription-job/*"
},
{
  "Sid": "TranscribeS3Storage",
  "Effect": "Allow",
  "Action": [
    "s3:PutObject",
    "s3:GetObject",
    "s3:DeleteObject"
  ],
  "Resource": "arn:aws:s3:::AWS_TRANSCRIBE_S3_BUCKET_VALUE/*"
}

Language Detection (Optional)¶

Environment Variables: AWS_COMPREHEND_REGION

Required for automatic language detection (used by TTS for voice selection).

{
  "Sid": "ComprehendLanguageDetection",
  "Effect": "Allow",
  "Action": [
    "comprehend:DetectDominantLanguage"
  ],
  "Resource": "*"
}

Text Translation (Optional)¶

Environment Variables: AWS_TRANSLATE_REGION

Required for text translation features.

{
  "Sid": "TranslateTextTranslation",
  "Effect": "Allow",
  "Action": [
    "translate:TranslateText"
  ],
  "Resource": "*"
}

API Key Authentication (Optional)¶

Required if you configure API authentication. See Authentication configuration section.

SSM Parameter Store¶

Environment Variables: API_KEY_SSM_PARAMETER

{
  "Sid": "SSMParameterAccess",
  "Effect": "Allow",
  "Action": [
    "ssm:GetParameter"
  ],
  "Resource": "arn:aws:ssm:REGION:ACCOUNT_ID:parameter/API_KEY_SSM_PARAMETER_VALUE"
}

{
  "Sid": "KMSDecryptionForSSM",
  "Effect": "Allow",
  "Action": [
    "kms:Decrypt"
  ],
  "Resource": "arn:aws:kms:REGION:ACCOUNT_ID:key/YOUR_KMS_KEY_ID",
  "Condition": {
    "StringEquals": {
      "kms:ViaService": "ssm.REGION.amazonaws.com"
    }
  }
}

Secrets Manager¶

Environment Variables: API_KEY_SECRETSMANAGER_SECRET

{
  "Sid": "SecretsManagerAccess",
  "Effect": "Allow",
  "Action": [
    "secretsmanager:GetSecretValue"
  ],
  "Resource": "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:API_KEY_SECRETSMANAGER_SECRET_VALUE"
}

Complete Policy Examples¶

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BedrockModelInvoke",
      "Effect": "Allow",
      "Action": [
        "bedrock:CountTokens",
        "bedrock:GetAsyncInvoke",
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:InvokeTool"
      ],
      "Resource": "*"
    },
    {
      "Sid": "BedrockAsyncInvokeTagging",
      "Effect": "Allow",
      "Action": [
        "bedrock:TagResource"
      ],
      "Resource": "arn:aws:bedrock:*:*:async-invoke/*"
    },
    {
      "Sid": "BedrockModelDiscovery",
      "Effect": "Allow",
      "Action": [
        "bedrock:ListFoundationModels",
        "bedrock:GetFoundationModelAvailability",
        "bedrock:ListProvisionedModelThroughputs",
        "bedrock:ListInferenceProfiles"
      ],
      "Resource": "*"
    },
    {
      "Sid": "BedrockMarketplaceAutoSubscribe",
      "Effect": "Allow",
      "Action": [
        "aws-marketplace:Subscribe",
        "aws-marketplace:ViewSubscriptions"
      ],
      "Resource": "*"
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BedrockModelInvoke",
      "Effect": "Allow",
      "Action": [
        "bedrock:CountTokens",
        "bedrock:GetAsyncInvoke",
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:InvokeTool"
      ],
      "Resource": "*"
    },
    {
      "Sid": "BedrockAsyncInvokeTagging",
      "Effect": "Allow",
      "Action": [
        "bedrock:TagResource"
      ],
      "Resource": "arn:aws:bedrock:*:*:async-invoke/*"
    },
    {
      "Sid": "BedrockModelDiscovery",
      "Effect": "Allow",
      "Action": [
        "bedrock:ListFoundationModels",
        "bedrock:GetFoundationModelAvailability",
        "bedrock:ListProvisionedModelThroughputs",
        "bedrock:ListInferenceProfiles"
      ],
      "Resource": "*"
    },
    {
      "Sid": "BedrockMarketplaceAutoSubscribe",
      "Effect": "Allow",
      "Action": [
        "aws-marketplace:Subscribe",
        "aws-marketplace:ViewSubscriptions"
      ],
      "Resource": "*"
    },
    {
      "Sid": "S3FileStorage",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:PutObjectTagging",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:CreateMultipartUpload",
        "s3:UploadPart",
        "s3:CompleteMultipartUpload",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": "arn:aws:s3:::my-stdapi-bucket/*"
    },
    {
      "Sid": "S3FileStorageList",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": "arn:aws:s3:::my-stdapi-bucket"
    },
    {
      "Sid": "SSMParameterAccess",
      "Effect": "Allow",
      "Action": [
        "ssm:GetParameter"
      ],
      "Resource": "arn:aws:ssm:us-east-1:123456789012:parameter/stdapi/prod/api-key"
    }
  ]
}

Permission Notes¶

Feature-Specific Permission Requirements¶

IAM Role vs. IAM User¶

stdapi.ai supports both IAM roles and IAM users:

• IAM Role (Recommended): Use when running on EC2, ECS, Lambda, or other AWS compute services. Attach the policy to the instance/task role.
• IAM User: Use when running outside AWS or for development. Create an IAM user with the required permissions and configure AWS credentials via environment variables or AWS CLI configuration.

Authentication¶

stdapi.ai supports three methods for API key authentication.

Method 1: SSM Parameter Store (Recommended)¶

Recommended - Use AWS Systems Manager Parameter Store for secure key storage with encryption, access control, and auditing. This method should be used only with already existing parameters.

API_KEY_SSM_PARAMETER¶

Purpose : Name of the SSM parameter containing the API key. The parameter is retrieved from the current region detected by the running container, or defaults to the first region in AWS_BEDROCK_REGIONS.

Recommendation : Use SecureString type for encryption at rest

IAM Permissions Required : ssm:GetParameter, kms:Decrypt (if encrypted)

export API_KEY_SSM_PARAMETER=/stdapi/prod/api-key

Method 2: Secrets Manager¶

Use AWS Secrets Manager for secure key storage with automatic rotation support. This method should be used only with already existing secrets.

API_KEY_SECRETSMANAGER_SECRET¶

Purpose : Name of the Secrets Manager secret containing the API key. The secret is retrieved from the current region detected by the running container, or defaults to the first region in AWS_BEDROCK_REGIONS.

Format : Can be a plain string or JSON object

IAM Permissions Required : secretsmanager:GetSecretValue

API_KEY_SECRETSMANAGER_KEY¶

Purpose : JSON key name within the secret (if the secret is a JSON object)

Default : api_key

Plain String Secret:

export API_KEY_SECRETSMANAGER_SECRET=stdapi-api-key

JSON Secret:

export API_KEY_SECRETSMANAGER_SECRET=stdapi-credentials
export API_KEY_SECRETSMANAGER_KEY=api_key

Example JSON secret structure:

{
  "api_key": "sk-1234567890abcdef...",
  "other_config": "value"
}

Method 3: Direct API Key¶

Provide the API key directly via environment variable.

API_KEY¶

Purpose : Static API key value

Security Warning : Avoid hardcoding in configuration files; use environment variables only

Client Usage : Clients must include this key in the Authorization: Bearer <key> header or X-API-Key header

export API_KEY=sk-1234567890abcdef...

API Compatibility¶

Configure the base URL paths for OpenAI and Anthropic-compatible API routes.

OPENAI_ROUTES_PREFIX¶

Purpose : Base path prefix for OpenAI-compatible API routes

Default : `` (empty, routes mounted at root)

Effect : All OpenAI-compatible endpoints will be mounted under this prefix

export OPENAI_ROUTES_PREFIX=/api

ANTHROPIC_ROUTES_PREFIX¶

Purpose : Base path prefix for Anthropic-compatible API routes

Default : /anthropic

Effect : All Anthropic-compatible endpoints will be mounted under this prefix

export ANTHROPIC_ROUTES_PREFIX=/anthropic

export ANTHROPIC_ROUTES_PREFIX=/api/anthropic

CORS Configuration¶

Configure Cross-Origin Resource Sharing (CORS) to control which web origins can access your API from browsers.

CORS_ALLOW_ORIGINS¶

Purpose : List of origins allowed to make cross-origin requests

Format : JSON array of origin URLs

Default : None (CORS not enabled)

Best Practice : Only enable if your API is accessed from web browsers; specify exact origins in production

# Not configured (default) - CORS middleware not enabled
# Browser cross-origin requests will be blocked
# No environment variable needed

# Development: Allow all origins
export CORS_ALLOW_ORIGINS='["*"]'

# Production: Specific origins only
export CORS_ALLOW_ORIGINS='["https://myapp.com", "https://app.example.com"]'

# Multiple environments
export CORS_ALLOW_ORIGINS='["https://app.example.com", "https://staging.example.com"]'

Trusted Host Configuration¶

Configure Host header validation to protect against Host header injection attacks.

TRUSTED_HOSTS¶

Purpose : List of trusted Host header values for validation

Format : JSON array of hostnames (supports wildcards)

Default : None (no Host header validation)

Best Practice : Use AWS ALB host-based routing rules instead when possible for better performance and management

# Not configured (default) - no Host header validation
# No environment variable needed

# Production: Specific hosts only
export TRUSTED_HOSTS='["api.example.com", "www.example.com"]'

# With wildcard subdomains
export TRUSTED_HOSTS='["*.example.com", "api.myapp.com"]'

# Multiple environments including localhost
export TRUSTED_HOSTS='["api.example.com", "staging.example.com", "localhost"]'

export TRUSTED_HOSTS='["api.example.com"]'

export TRUSTED_HOSTS='["*.example.com", "*.myapp.com", "api.production.com"]'

export TRUSTED_HOSTS='["api.example.com", "localhost", "127.0.0.1"]'

aws elbv2 create-rule \
  --listener-arn arn:aws:elasticloadbalancing:... \
  --priority 1 \
  --conditions Field=host-header,Values=api.example.com \
  --actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:...

Proxy Headers Configuration¶

Configure X-Forwarded-* header processing when running behind reverse proxies or load balancers.

ENABLE_PROXY_HEADERS¶

Purpose : Enable trusting X-Forwarded-* headers from reverse proxies

Type : Boolean

Default : false (disabled)

Best Practice : Only enable when running behind a trusted reverse proxy

# Disabled (default) - do not trust X-Forwarded-* headers
# No environment variable needed

# Enable when behind reverse proxy
export ENABLE_PROXY_HEADERS=true

# Do NOT enable proxy headers
# ENABLE_PROXY_HEADERS should remain false (default)

export ENABLE_PROXY_HEADERS=true

export ENABLE_PROXY_HEADERS=true

TLS / SSL Configuration¶

Configure end-to-end TLS encryption within the container. These are native Granian environment variables and are available with the provided container images.

GRANIAN_SSL_CERTIFICATE¶

Purpose : Path to the SSL certificate file

Type : File path

GRANIAN_SSL_KEYFILE¶

Purpose : Path to the SSL private key file (PKCS#8 format only)

Type : File path

GRANIAN_SSL_KEYFILE_PASSWORD¶

Purpose : Password for the private key file

Type : String

GRANIAN_SSL_PROTOCOL_MIN¶

Purpose : Minimum supported TLS version (tls1.2 or tls1.3)

Type : Enum

Default : tls1.3

GRANIAN_SSL_CA¶

Purpose : Path to the CA certificate bundle used to verify client certificates (mTLS)

Type : File path

GRANIAN_SSL_CLIENT_VERIFY¶

Purpose : Enable client certificate verification (mTLS)

Type : Boolean

Default : false (disabled)

GZip Compression¶

Configure automatic GZip compression for HTTP responses to reduce bandwidth usage and improve response times.

ENABLE_GZIP¶

Purpose : Enable GZip compression for HTTP responses

Type : Boolean

Default : false (disabled)

Best Practice : Use AWS ALB or CloudFront compression instead when available for better performance

# Disabled (default) - no response compression
# No environment variable needed

# Enable GZip compression (responses larger than 1 KiB will be compressed)
export ENABLE_GZIP=true

aws elbv2 modify-target-group-attributes \
  --target-group-arn arn:aws:elasticloadbalancing:region:account:targetgroup/... \
  --attributes Key=compression.enabled,Value=true

aws cloudfront update-distribution \
  --id YOUR_DISTRIBUTION_ID \
  --distribution-config file://config.json
# Set "Compress": true in the distribution config

SSRF Protection¶

Configure Server-Side Request Forgery (SSRF) protection to prevent unauthorized access to internal networks.

SSRF_PROTECTION_BLOCK_PRIVATE_NETWORKS¶

Purpose : Enable SSRF protection by blocking requests to private/local networks

Type : Boolean

Default : true (enabled for security)

Best Practice : Keep enabled in production to protect against SSRF attacks

# Enabled (default) - block private networks
# No environment variable needed

# Disable only in controlled environments that need local network access
export SSRF_PROTECTION_BLOCK_PRIVATE_NETWORKS=false

Observability (OpenTelemetry)¶

Configure distributed tracing for debugging and performance monitoring. stdapi.ai integrates with AWS X-Ray, Jaeger, DataDog, and other OTLP-compatible systems.

OTEL_ENABLED¶

Purpose : Enable or disable OpenTelemetry tracing

Type : Boolean

Default : false

export OTEL_ENABLED=true

OTEL_SERVICE_NAME¶

Purpose : Service identifier in trace visualizations

Default : stdapi

Best Practice : Use descriptive names with environment information

export OTEL_SERVICE_NAME=stdapi-production-us-east-1

OTEL_EXPORTER_ENDPOINT¶

Purpose : OTLP HTTP endpoint URL for sending traces

Default : http://127.0.0.1:4318/v1/traces

Protocol : Must support OTLP HTTP format

AWS X-Ray (via ADOT):

export OTEL_EXPORTER_ENDPOINT=http://127.0.0.1:4318/v1/traces

Jaeger:

export OTEL_EXPORTER_ENDPOINT=http://jaeger:14268/api/traces

Cloud Provider OTLP:

# Use provider-specific OTLP endpoints
export OTEL_EXPORTER_ENDPOINT=https://your-provider-otlp-endpoint.com/v1/traces

OTEL_SAMPLE_RATE¶

Purpose : Percentage of requests to trace (controls cost vs. observability)

Type : Float (0.0 to 1.0)

Default : 1.0 (100%)

Development:

# Trace everything for debugging
export OTEL_SAMPLE_RATE=1.0

Production (Moderate Traffic):

# Sample 10% of requests
export OTEL_SAMPLE_RATE=0.1

Production (High Traffic):

# Sample 1% of requests
export OTEL_SAMPLE_RATE=0.01

API Documentation Routes¶

stdapi.ai provides automatic API documentation routes, which are disabled by default for security in production environments.

ENABLE_DOCS¶

Purpose : Enable interactive Swagger UI documentation at /docs

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_DOCS=true

ENABLE_REDOC¶

Purpose : Enable ReDoc documentation UI at /redoc

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_REDOC=true

ENABLE_OPENAPI_JSON¶

Purpose : Enable OpenAPI schema JSON endpoint at /openapi.json

Type : Boolean

Default : false (disabled)

# Enable for development
export ENABLE_OPENAPI_JSON=true

Development Configuration¶

Enable all documentation routes for local development:

export ENABLE_DOCS=true
export ENABLE_REDOC=true
# ENABLE_OPENAPI_JSON is automatically enabled when ENABLE_DOCS or ENABLE_REDOC is true

Or enable only Swagger UI:

export ENABLE_DOCS=true
# ENABLE_OPENAPI_JSON is automatically enabled

Or enable only ReDoc:

export ENABLE_REDOC=true
# ENABLE_OPENAPI_JSON is automatically enabled

Production Best Practice¶

# Keep all routes disabled in production (default)
# No environment variables needed - defaults to false

Validation and Logging¶

For comprehensive logging and monitoring information, see the Logging and Monitoring guide.

Strict Validation:

# Returns HTTP 400 for requests with unexpected fields
export STRICT_INPUT_VALIDATION=true

Logging Level¶

LOG_LEVEL¶

Purpose : Control the minimum severity of log events written to STDOUT

Default : info

Options : info, warning, error, critical, disabled

Behavior : Only log events at or above the configured level are output. Log levels are ordered by severity: info < warning < error < critical

# Default: Output all log events
export LOG_LEVEL=info

# Production: Suppress info logs, show only warnings and higher
export LOG_LEVEL=warning

# Critical only: Show only critical errors
export LOG_LEVEL=critical

# Disable logging: Suppress all log output (not recommended)
export LOG_LEVEL=disabled

Debug Logging:

# Enable for debugging (NOT recommended for production)
export LOG_REQUEST_PARAMS=true

Client IP Logging¶

LOG_CLIENT_IP¶

Purpose : Enable logging of client IP addresses for each request and add IP to OpenTelemetry spans

Type : Boolean

Default : false (disabled for privacy)

# Disabled (default) - no client IP logging
# No environment variable needed

# Enable client IP logging
export LOG_CLIENT_IP=true

# Enable proxy headers to get real client IPs
export ENABLE_PROXY_HEADERS=true
# Enable client IP logging
export LOG_CLIENT_IP=true

# Only enable client IP logging
export LOG_CLIENT_IP=true
# ENABLE_PROXY_HEADERS remains false (default)

Timezone Configuration:

# UTC (default)
export TIMEZONE=UTC

# North America
export TIMEZONE=America/New_York

# Europe
export TIMEZONE=Europe/London

Bedrock Guardrails¶

Amazon Bedrock Guardrails add content filtering and safety controls to model inputs and outputs.

Global Configuration¶

AWS_BEDROCK_GUARDRAIL_IDENTIFIER¶

Purpose : ID of the Bedrock Guardrail to apply

Required : Yes (together with AWS_BEDROCK_GUARDRAIL_VERSION)

export AWS_BEDROCK_GUARDRAIL_IDENTIFIER=abc123def456

AWS_BEDROCK_GUARDRAIL_VERSION¶

Purpose : Version of the Bedrock Guardrail

Required : Yes (together with AWS_BEDROCK_GUARDRAIL_IDENTIFIER)

export AWS_BEDROCK_GUARDRAIL_VERSION=1

AWS_BEDROCK_GUARDRAIL_TRACE¶

Purpose : Trace level for guardrail evaluation

Options : disabled, enabled, enabled_full

Default : None (optional)

export AWS_BEDROCK_GUARDRAIL_TRACE=enabled

AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE¶

Purpose : Control whether users can override the global guardrail configuration at request level via HTTP headers

Default : false (disabled for security)

Security Consideration : When set to false (default) and a global guardrail is configured, only the global configuration is enforced, preventing users from bypassing or modifying safety controls. Set to true if you need to allow per-request guardrail customization to override the global configuration.

Auto-Enable Behavior : If no global guardrail configuration is set (both AWS_BEDROCK_GUARDRAIL_IDENTIFIER and AWS_BEDROCK_GUARDRAIL_VERSION are unset), this setting is automatically set to true at startup, allowing per-request guardrails when no global policy is enforced.

export AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE=true

export AWS_BEDROCK_GUARDRAIL_IDENTIFIER=abc123def456
export AWS_BEDROCK_GUARDRAIL_VERSION=1
export AWS_BEDROCK_GUARDRAIL_TRACE=enabled
export AWS_BEDROCK_ALLOW_GUARDRAIL_OVERRIDE=false  # Default: prevent overrides

Per-Request Configuration¶

Use HTTP headers to specify guardrail settings per request:

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-GuardrailIdentifier: abc123def456" \
  -H "X-Amzn-Bedrock-GuardrailVersion: 1" \
  -H "X-Amzn-Bedrock-Trace: enabled" \
  -d '{"model": "anthropic.claude-3-sonnet", "messages": [...]}'

Request Body Configuration¶

The amazon-bedrock-guardrailConfig object in the request body is supported for OpenAI Chat Completions compatibility.

Bedrock Service Tier and Performance Configuration¶

Amazon Bedrock service tiers and performance configurations allow you to optimize AI workload performance and cost trade-offs. Configure latency optimization and throughput priority for your inference requests.

Service Tiers¶

Service tiers help you match AI workload performance with cost by selecting the appropriate throughput and latency characteristics:

• priority - Highest priority processing with guaranteed capacity and fastest response times. Best for latency-sensitive applications.
• default - Standard processing with balanced performance and cost. Suitable for most production workloads.
• flex - Cost-optimized processing with flexible scheduling. Best for batch jobs and non-time-sensitive workloads.

Performance Configuration¶

Performance configuration allows you to optimize for latency:

• standard - Standard latency profile with balanced performance
• optimized - Optimized for lowest possible latency

Per-Request Configuration¶

Configure service tier and performance settings per request using HTTP headers. These headers are available on all Bedrock-based routes (Chat Completions, Embeddings, Images):

curl -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: priority" \
  -H "X-Amzn-Bedrock-PerformanceConfig-Latency: optimized" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-sonnet-4-5-20250929-v1:0",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

curl -X POST https://api.example.com/v1/embeddings \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: flex" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-2-multimodal-embeddings-v1:0",
    "input": ["text 1", "text 2", "text 3"]
  }'

curl -X POST https://api.example.com/v1/images/generations \
  -H "Authorization: Bearer sk-..." \
  -H "X-Amzn-Bedrock-Service-Tier: default" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "amazon.nova-canvas-v1:0",
    "prompt": "A serene mountain landscape"
  }'

Audio and Text-to-Speech¶

DEFAULT_TTS_MODEL¶

Purpose : Default text-to-speech model when not specified in requests

Default : amazon.polly-standard

export DEFAULT_TTS_MODEL=amazon.polly-neural

DEFAULT_TTS_LANGUAGE¶

Purpose : Default language code for text-to-speech synthesis when using OpenAI voice names

Default : None (automatic language detection via AWS Comprehend)

Behavior : When specified, this language is used instead of automatic detection. When not set, AWS Comprehend detects the language automatically from the input text.

Valid Language Codes: Any AWS Polly language code (e.g., en-US, fr-FR, es-ES, de-DE, ja-JP)

# Use English (US) for all TTS requests
export DEFAULT_TTS_LANGUAGE=en-US

# Use French for all TTS requests
export DEFAULT_TTS_LANGUAGE=fr-FR

Token Counting¶

Control how token usage is calculated and reported in API responses.

TOKENS_ESTIMATION¶

Purpose : Estimate token counts using a tokenizer when the model doesn't return them directly

Type : Boolean

Default : false

export TOKENS_ESTIMATION=true

TOKENS_ESTIMATION_DEFAULT_ENCODING¶

Purpose : Tiktoken encoding algorithm for token estimation

Default : o200k_base

export TOKENS_ESTIMATION_DEFAULT_ENCODING=o200k_base

Model Cache¶

stdapi.ai automatically discovers and caches available Bedrock models from configured regions. The cache is refreshed on-demand when expired, not via background tasks.

MODEL_CACHE_SECONDS¶

Purpose : Cache lifetime for the Bedrock models list before refresh

Type : Integer (seconds)

Default : 900 (15 minutes)

Behavior : When a request needs the model list (e.g., model lookup, /models endpoint) and the cache has expired, the server queries AWS Bedrock to discover newly available models, check for model access changes, and update inference profile configurations. This cache also applies to application inference profile and prompt router information when users pass ARNs directly (if enabled via AWS_BEDROCK_ALLOW_APPLICATION_INFERENCE_PROFILE_ARN or AWS_BEDROCK_ALLOW_PROMPT_ROUTER_ARN)

# Default: 15 minutes
export MODEL_CACHE_SECONDS=900

# More frequent updates (5 minutes)
export MODEL_CACHE_SECONDS=300

# Less frequent updates (1 hour)
export MODEL_CACHE_SECONDS=3600

AI_RESPONSE_TIMEOUT¶

Purpose : Maximum time in seconds to wait for an AI model to complete a response

Type : Integer (seconds, must be greater than 0)

Default : 600 (10 minutes)

Behavior : Applies to both streaming and non-streaming requests. The timer starts from the moment the model begins generating and covers the full duration until the last token is received. If the model does not complete within this limit, the connection is closed and the request fails with a timeout error

# Default (10 minutes) - suitable for extended thinking models
export AI_RESPONSE_TIMEOUT=600

# Shorter timeout for standard models (2 minutes)
export AI_RESPONSE_TIMEOUT=120

# Longer timeout for very long documents or high reasoning budgets (15 minutes)
export AI_RESPONSE_TIMEOUT=900

Default Model Parameters¶

Configure default inference parameters applied automatically to specific models.

DEFAULT_MODEL_PARAMS¶

Purpose : Per-model default parameters

Format : JSON object with model IDs as keys

Supported Parameters:

Configuration Examples¶

Basic Parameters:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-micro-v1:0": {
    "temperature": 0.3,
    "max_tokens": 800
  }
}'

Provider-Specific Features:

export DEFAULT_MODEL_PARAMS='{
  "anthropic.claude-sonnet-4-5-20250929-v1:0": {
    "anthropic_beta": ["Interleaved-thinking-2025-05-14"]
  }
}'

Multiple Models:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-micro-v1:0": {
    "temperature": 0.3,
    "max_tokens": 500
  },
  "amazon.nova-lite-v1:0": {
    "temperature": 0.7,
    "max_tokens": 2000
  },
  "anthropic.claude-sonnet-4-5-20250929-v1:0": {
    "temperature": 0.5,
    "top_p": 0.9,
    "anthropic_beta": ["Interleaved-thinking-2025-05-14"]
  }
}'

Advanced Configuration:

export DEFAULT_MODEL_PARAMS='{
  "amazon.nova-pro-v1:0": {
    "temperature": 0.7,
    "top_p": 0.95,
    "max_tokens": 4096,
    "stop_sequences": ["Human:", "Assistant:"]
  }
}'

Parameter Merging¶

graph LR
    A[Default Parameters] --> B[Merged Config]
    C[Request Parameters] --> B
    B --> D[Final Configuration]

1. Default parameters are applied first (from DEFAULT_MODEL_PARAMS)
2. Request parameters override defaults if both are specified
3. Provider-specific fields are forwarded to Bedrock as additional model request fields
4. Unsupported fields that would change output cause HTTP 400 error; otherwise ignored

Model Aliases¶

Configure custom aliases to map user-friendly model names to actual model IDs. This enables OpenAI API compatibility and simplifies model references.

MODEL_ALIASES¶

Purpose : Map alias names to actual model IDs or ARNs

Format : JSON object with alias names as keys and model IDs or ARNs as values

Default : {} (empty, uses built-in defaults only)

Configuration Examples¶

Basic Alias:

export MODEL_ALIASES='{
  "my-tts": "amazon.polly-neural",
  "my-stt": "amazon.transcribe"
}'

Override Default Aliases:

# Override the default tts-1 mapping
export MODEL_ALIASES='{
  "tts-1": "amazon.polly-generative"
}'

Multiple Custom Aliases:

export MODEL_ALIASES='{
  "fast-model": "amazon.nova-micro-v1:0",
  "balanced-model": "amazon.nova-lite-v1:0",
  "quality-model": "amazon.nova-pro-v1:0",
  "claude": "anthropic.claude-sonnet-4-5-20250929-v1:0"
}'

Map OpenAI Models to Bedrock:

# Make OpenAI model names work with AWS Bedrock models
export MODEL_ALIASES='{
  "gpt-5": "anthropic.claude-sonnet-4-5-20250929-v1:0",
  "gpt-4o": "anthropic.claude-3-5-sonnet-20241022-v2:0",
  "gpt-4o-mini": "anthropic.claude-3-5-haiku-20241022-v1:0",
  "dall-e-3": "amazon.nova-canvas-v1:0",
  "dall-e-2": "stability.stable-image-ultra-v1:0"
}'

Override Deprecated Models:

# Redirect deprecated model IDs to their newer replacements
export MODEL_ALIASES='{
  "amazon.titan-image-generator-v1": "amazon.nova-canvas-v1:0",
  "amazon.titan-text-express-v1": "amazon.nova-lite-v1:0",
  "anthropic.claude-3-5-sonnet-20240620-v1:0": "anthropic.claude-sonnet-4-5-20250929-v1:0",
  "stability.stable-image-ultra-v1:0": "stability.stable-image-ultra-v1:1"
}'

Advanced Routing with ARNs:

# Map friendly names to Application Inference Profiles or Prompt Routers
export MODEL_ALIASES='{
  "my-router": "arn:aws:bedrock:us-east-1:123456789012:default-prompt-router/cost-optimizer",
  "my-profile": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123xyz",
}'

Using Aliases in API Requests¶

Once configured, aliases can be used anywhere a model ID is expected:

# Using the default tts-1 alias
curl https://api.example.com/v1/audio/speech \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tts-1",
    "input": "Hello world",
    "voice": "alloy"
  }'

# Using a custom alias
curl https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fast-model",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Alias Resolution¶

graph LR
    A[API Request] --> B{Alias Exists?}
    B -->|Yes| C[Resolve to Model ID]
    B -->|No| D[Use as Model ID]
    C --> E[Model Validation]
    D --> E
    E --> F[Execute Request]

1. User-configured aliases override default aliases
2. Default aliases apply if not overridden
3. Non-aliased names pass through unchanged
4. Resolved model ID is validated and used for the request

System Prompt Handling¶

Control how system prompts are handled for models that don't support them.

DROP_UNSUPPORTED_SYSTEM_PROMPT¶

Purpose : Control system prompt behavior for models that don't support system prompts