Logging and Monitoring¶

stdapi.ai provides production observability. It emits structured JSON logs for every request, stream, and background task, and integrates with OpenTelemetry (OTel) for traces and metrics. This guide shows how to enable observability, read the logs, and correlate signals across systems.

Quick start (2 minutes)¶

Set these environment variables, then restart the service (see the Configuration Guide for details):

# Set minimum log level (optional, defaults to "info")
# Options: info, warning, error, critical, disabled
export LOG_LEVEL=warning

# Enable OpenTelemetry tracing
export OTEL_ENABLED=true
export OTEL_SERVICE_NAME=stdapi
# 0.0–1.0 (10% example)
export OTEL_SAMPLE_RATE=0.1

# Include request/response payloads in logs (for debugging ONLY)
export LOG_REQUEST_PARAMS=true

# Log client IP addresses (requires ENABLE_PROXY_HEADERS for real client IPs)
export LOG_CLIENT_IP=true
export ENABLE_PROXY_HEADERS=true  # When behind ALB/CloudFront

Event types¶

stdapi.ai emits five kinds of JSON events (one per line):

Common fields¶

Each event shares core fields and may add type‑specific ones.

Correlating logs and traces¶

• Group events by id to reconstruct a full request lifecycle (request → stream(s) → background).
• The x-request-id response header exposes the same value so external systems can propagate correlation.
• With OTel enabled, a root span named like POST /v1/... is created and carries attributes: http.method, http.url, http.user_agent, request.id, server.id, http.status_code, and duration_ms.

Reading the logs (what to look for)¶

• High latency: Inspect execution_time_ms on the request event. If the response was streamed, also sum request_stream durations. Combine with OTel spans to locate downstream delays (model provider, S3, etc.).
• Errors: Look for level=critical and error_detail (formatted exceptions). With OTel, the span is marked error with attributes error=true and error.message.

• Payload issues: Temporarily enable LOG_REQUEST_PARAMS=true to validate requests/responses, then disable.
• Client identification: client_user_agent and optional request_user_id / request_org_id help tie requests to users.
• Routing confirmation: model_id and voice_id confirm which provider/model/voice handled the request.

Controlling log verbosity¶

The LOG_LEVEL environment variable controls which log events are written to STDOUT. Set it to filter out lower-severity events. For detailed configuration options, see the Logging Level section in the Configuration Guide.

• info (default): All events are logged (info, warning, error, critical)
• warning: Only warnings and higher severity (warning, error, critical) - recommended for production
• error: Only errors and critical events
• critical: Only critical events
• disabled: No log output (not recommended)

# Production example: reduce log volume while maintaining visibility
export LOG_LEVEL=warning

OpenTelemetry integration¶

When OTEL_ENABLED=true:

• A span is created per request and for streaming/background blocks.
• Spans carry request.id and server.id for correlation.
• 4xx/5xx status_code marks the span with an error status.
• Sampling is controlled via OTEL_SAMPLE_RATE.

For exporters and advanced setup, rely on standard OTel environment variables supported by your exporter/backend.

Example events¶

Example — Request with payload logging enabled

{
  "type": "request",
  "level": "info",
  "date": "2025-01-01T12:00:00Z",
  "server_id": "stdapi-1",
  "id": "a1b2c3d4",
  "method": "POST",
  "path": "/v1/chat/completions",
  "status_code": 200,
  "model_id": "anthropic.claude-sonnet-4-5-20250929-v1:0",
  "execution_time_ms": 842,
  "request_params": {"messages": [{"role": "user", "content": "..."}]},
  "request_response": {"id": "cmpl_...", "choices": [...], "usage": {...}}
}

Example — Streaming segment (SSE/audio)

{
  "type": "request_stream",
  "level": "info",
  "date": "2025-01-01T12:00:01Z",
  "server_id": "stdapi-1",
  "id": "a1b2c3d4",
  "execution_time_ms": 1234
}

Example — Background work correlated to a request

{
  "type": "background",
  "level": "info",
  "date": "2025-01-01T12:00:02Z",
  "server_id": "stdapi-1",
  "id": "a1b2c3d4",
  "event": "image-upload-s3",
  "execution_time_ms": 97
}

Example — Error with captured details

{
  "type": "request",
  "level": "critical",
  "date": "2025-01-01T12:00:05Z",
  "server_id": "stdapi-1",
  "id": "e9f0a1b2",
  "method": "POST",
  "path": "/v1/images/edits",
  "status_code": 500,
  "error_detail": ["Traceback (most recent call last): ..."],
  "execution_time_ms": 12
}

CloudWatch Logs Insights: ready‑to‑use queries¶

These examples assume JSON logs in CloudWatch Logs (default with ECS awslogs/awsfirelens). Adjust the log group and time range.

1) Follow a specific request across request/stream/background¶

fields @timestamp, type, level, path, event, status_code, execution_time_ms
| filter id = "<paste-request-id>"
| sort @timestamp asc

2) Find recent errors with context¶

fields @timestamp, level, type, path, status_code, id, error_detail
| filter level in ["error", "critical"]
| sort @timestamp desc
| limit 100

3) High-latency endpoints (P95/P99)¶

fields path, execution_time_ms
| filter type = "request" and ispresent(execution_time_ms)
| stats pct(execution_time_ms, 95) as p95_ms, pct(execution_time_ms, 99) as p99_ms, avg(execution_time_ms) as avg_ms by path
| sort p95_ms desc

AWS service-level logs and metrics¶

Beyond stdapi.ai logs and OTel traces, use AWS-native signals from the underlying AI services to validate provider behavior, monitor throttling/latency, and audit access. Enable only what you need: some options can capture content and increase costs. For full, up-to-date details, refer to the official AWS documentation for more information.

• CloudWatch Metrics: Throughput, latency, throttling, and error rates per service/region.
• CloudTrail: Control-plane auditing of API calls (who did what, when, from where).
• Content/Invocation logging: Optional features that may record inputs/outputs. Use with caution and encryption/retention controls.
• Correlation: Service logs won’t include StdAPI x-request-id. Correlate by time window, region, model/voice/job identifiers, and volume. Use StdAPI model_id, voice_id, and execution_time_ms to narrow windows.
• AWS Bedrock Invocation logging (optional): Export invocation metadata and, if enabled, content to CloudWatch Logs/S3/Firehose. Treat prompts/completions as sensitive; manage retention and KMS.

Troubleshooting checklist¶

• No logs visible: Ensure you are reading container STDOUT. On ECS/Kubernetes, verify the log driver and retention.
• Missing request_params: Confirm LOG_REQUEST_PARAMS=true and restart after changing environment variables.
• No traces: Verify OTEL_ENABLED=true and that exporters are configured and reachable.
• Correlation missed: Ensure clients read and propagate x-request-id for multi‑hop requests.