Observability and Usage¶

What the gateway reports about itself: logs, traces, metrics, per-request cost and the usage endpoints. Part of the Configuration Guide.

Settings Summary¶

Logging¶

CloudWatch Metrics¶

Cost Tracking¶

Usage API¶

Observability (OpenTelemetry)¶

Observability (OpenTelemetry)¶

Configure distributed tracing for debugging and performance monitoring. stdapi.ai integrates with AWS X-Ray, Jaeger, DataDog, and other OTLP-compatible systems.

OTEL_ENABLED¶

Purpose : Enable or disable OpenTelemetry tracing

Type : Boolean

Default : false

export OTEL_ENABLED=true

OTEL_SERVICE_NAME¶

Purpose : Service identifier in trace visualizations

Default : stdapi.ai

Best Practice : Use descriptive names with environment information

export OTEL_SERVICE_NAME=stdapi-production-us-east-1

OTEL_EXPORTER_ENDPOINT¶

Purpose : OTLP HTTP endpoint URL for sending traces

Default : http://127.0.0.1:4318/v1/traces

Protocol : Must support OTLP HTTP format

AWS X-Ray (via ADOT):

export OTEL_EXPORTER_ENDPOINT=http://127.0.0.1:4318/v1/traces

Jaeger:

export OTEL_EXPORTER_ENDPOINT=http://jaeger:14268/api/traces

Cloud Provider OTLP:

# Use provider-specific OTLP endpoints
export OTEL_EXPORTER_ENDPOINT=https://your-provider-otlp-endpoint.com/v1/traces

OTEL_SAMPLE_RATE¶

Purpose : Percentage of requests to trace (controls cost vs. observability)

Type : Float (0.0 to 1.0)

Default : 1.0 (100%)

Development:

# Trace everything for debugging
export OTEL_SAMPLE_RATE=1.0

Production (Moderate Traffic):

# Sample 10% of requests
export OTEL_SAMPLE_RATE=0.1

Production (High Traffic):

# Sample 1% of requests
export OTEL_SAMPLE_RATE=0.01

Validation and Logging¶

The settings below decide what the gateway writes; the Logging and Monitoring guide describes what comes out of them — the event types and the fields common to all of them, the usage metrics carried on each entry, the CloudWatch EMF metric lines, and CloudWatch Logs Insights queries that follow a single request across its request, stream and background events.

STRICT_INPUT_VALIDATION¶

Purpose : Reject API requests containing unknown/extra fields instead of ignoring them

Type : Boolean

Default : false

# Returns HTTP 400 for requests with unexpected fields
export STRICT_INPUT_VALIDATION=true

CHAT_COMPLETIONS_REASONING_FIELD¶

Purpose : Choose which field carries a reasoning model's thinking text on /v1/chat/completions

Type : String

Default : reasoning_content

Options : reasoning_content, reasoning, none

Behavior : The OpenAI Chat Completions API returns no thinking text of its own — it reports only a reasoning_tokens count — so the providers that do return it have settled on two different names. reasoning_content is the DeepSeek spelling, which most clients that read reasoning at all look for first. reasoning is the name used by OpenRouter and vLLM. none emits neither, keeping responses strictly OpenAI-shaped. : The setting applies to both the completed message and the streamed deltas, so a client never sees one name while streaming and another at the end. Callers can also suppress reasoning per request with include_reasoning: false or reasoning: {"exclude": true}, whatever this is set to.

# Default: the name most clients read
export CHAT_COMPLETIONS_REASONING_FIELD=reasoning_content

# For clients written against OpenRouter or vLLM
export CHAT_COMPLETIONS_REASONING_FIELD=reasoning

# Strict OpenAI shape: never return thinking text
export CHAT_COMPLETIONS_REASONING_FIELD=none

LOG_LEVEL¶

Purpose : Control the minimum severity of log events written to STDOUT

Default : info

Options : info, warning, error, critical, disabled

Behavior : Only log events at or above the configured level are output. Log levels are ordered by severity: info < warning < error < critical

# Default: Output all log events
export LOG_LEVEL=info

# Production: Suppress info logs, show only warnings and higher
export LOG_LEVEL=warning

# Critical only: Show only critical errors
export LOG_LEVEL=critical

# Disable logging: Suppress all log output (not recommended)
export LOG_LEVEL=disabled

LOG_REQUEST_PARAMS¶

Purpose : Include request and response parameters (JSON body, form, query) in logs for integration debugging

Type : Boolean

Default : false

# Enable for debugging (NOT recommended for production)
export LOG_REQUEST_PARAMS=true

LOG_CLIENT_IP¶

Purpose : Enable logging of client IP addresses for each request and add IP to OpenTelemetry spans

Type : Boolean

Default : false (disabled for privacy)

# Disabled (default) - no client IP logging
# No environment variable needed

# Enable client IP logging
export LOG_CLIENT_IP=true

# Enable proxy headers to get real client IPs
export ENABLE_PROXY_HEADERS=true
# Enable client IP logging
export LOG_CLIENT_IP=true

# Only enable client IP logging
export LOG_CLIENT_IP=true
# ENABLE_PROXY_HEADERS remains false (default)

TIMEZONE¶

Purpose : IANA timezone identifier used for request date and time

Type : String (IANA timezone identifier)

Default : UTC

# UTC (default)
export TIMEZONE=UTC

# North America
export TIMEZONE=America/New_York

# Europe
export TIMEZONE=Europe/London

CloudWatch Metrics and Cost Tracking¶

The behavior of these settings — EMF line structure, cost log format, pricing accuracy, regional price fallback, known limitations, and the price override format with examples — is documented in CloudWatch Metrics (EMF) and Cost Tracking in the Logging and Monitoring guide.

CLOUDWATCH_METRICS¶

Purpose : Emit per-request AWS-billed usage as CloudWatch Embedded Metric Format (EMF) log lines

Type : Boolean

Default : false

export CLOUDWATCH_METRICS=true

CLOUDWATCH_METRICS_NAMESPACE¶

Purpose : CloudWatch namespace under which the emitted usage metrics are grouped

Type : String

Default : stdapi

Requirement : 1-255 characters, alphanumeric plus . - _ / # :, must not start with the reserved AWS/ prefix

export CLOUDWATCH_METRICS_NAMESPACE=my-app-metrics

CLOUDWATCH_METRICS_USER_DIMENSION¶

Purpose : Also publish the authenticated caller as a User metric dimension. This is what makes group_by=user_id answerable on the Usage API; without it, those queries have no per-user series to read.

Type : Boolean

Default : false

Requirement : CLOUDWATCH_METRICS and USAGE_API must both be enabled

export CLOUDWATCH_METRICS_USER_DIMENSION=true

CLOUDWATCH_METRICS_REGION¶

Purpose : Region the Usage API reads the published metrics from. Metrics are published in the region the server's logs are ingested in, which is the server's own region by default — set this only when the logs are shipped elsewhere, because a mismatch makes every usage query answer with empty buckets.

Type : String (AWS region name)

Default : The region the server runs in — its configured AWS region, or the first AWS_BEDROCK_REGIONS entry when none is set

export CLOUDWATCH_METRICS_REGION=eu-west-1

COST_TRACKING¶

Purpose : Estimate each request's cost from the published AWS price list — an estimate computed from billed quantities, not read back from your invoice (details and accuracy caveats). Disabled by default: it requires the extra pricing:GetProducts IAM permission — see Cost Tracking IAM Permissions.

Type : Boolean

Default : false

export COST_TRACKING=true

COST_PRICE_OVERRIDES¶

Purpose : Operator-supplied unit price overrides for models not covered by the AWS Price List API (format and example)

Type : JSON object — keys are model IDs, values are dicts mapping dimension name to price per one unit

Default : {}

Usage API¶

The Organization Usage and Costs API serves the OpenAI Administration usage and costs endpoints from the metrics this deployment already publishes. It is opt-in and off by default; while it is off, the routes still exist and answer 503, so a client can tell a disabled feature from a missing deployment.

The routes it adds, under the configured OPENAI_ROUTES_PREFIX:

/v1/organization/usage/completions
/v1/organization/usage/embeddings
/v1/organization/usage/moderations
/v1/organization/usage/images
/v1/organization/usage/audio_speeches
/v1/organization/usage/audio_transcriptions
/v1/organization/usage/web_search_calls
/v1/organization/usage/file_search_calls
/v1/organization/usage/vector_stores
/v1/organization/usage/code_interpreter_sessions
/v1/organization/costs

USAGE_API¶

Purpose : Serve the organization usage and costs endpoints. When disabled, the routes exist and answer 503. Enabling it also publishes the usage metrics under an additional dimension set, which carries its own monthly cost.

Type : Boolean

Default : false

Requirement : CLOUDWATCH_METRICS must be enabled, or startup fails; /v1/organization/costs additionally requires COST_TRACKING, which is only a startup warning when missing

export USAGE_API=true

USAGE_API_ADMIN_SCOPES¶

Purpose : OAuth 2.0 scopes an Amazon Cognito token must all carry to read these endpoints. Left empty, no user pool token is accepted and only the deployment's own API_KEY may read them. A tenant API key is never accepted, whatever this is set to — organization-wide usage is not a tenant's to read.

Type : Comma-separated list of scope names

Default : Empty — admin API key only

export USAGE_API_ADMIN_SCOPES=stdapi/admin,stdapi/usage.read

USAGE_API_MAX_METRICS¶

Purpose : Refuse a query that would read more metric series than this, before it is billed. A query over the limit is rejected rather than served, so a client narrows its group_by or its time range instead of running up the bill.

Type : Integer

Default : 500

Requirement : 1–500 — 500 is also the Amazon CloudWatch per-request maximum

# Tighter cap on a deployment with a large model catalogue
export USAGE_API_MAX_METRICS=100

USAGE_API_MAX_RANGE_DAYS¶

Purpose : Longest span allowed between start_time and end_time on a query. A longer range is rejected rather than served, so a client cannot ask for a year of daily buckets in one call.

Type : Integer (days)

Default : 92

Requirement : 1–455

# One month per query
export USAGE_API_MAX_RANGE_DAYS=31

USAGE_API_CACHE_TTL¶

Purpose : Seconds an answered query is reused for. This is what makes a polling client affordable: within the TTL, repeated identical queries are served from the cached answer and cost nothing. Set 0 to disable the cache — every query then reaches Amazon CloudWatch and is billed.

Type : Integer (seconds)

Default : 60

Requirement : 0–3600

# Longer reuse window for dashboards that poll aggressively
export USAGE_API_CACHE_TTL=300