Logging & Monitoring¶

stdapi.ai provides production-grade observability out of the box. Track request performance, debug issues, monitor costs, and ensure compliance with structured JSON logging and OpenTelemetry integration.

Why this matters:

• Troubleshoot issues fast - Structured logs with request IDs, timings, and error details
• Monitor costs - Track model usage, request volume, and performance by endpoint
• Ensure compliance - Full audit trail with client IPs, user IDs, and request/response logging
• Performance optimization - Identify slow requests, high-latency endpoints, bottlenecks
• AWS integration - Native CloudWatch Logs, X-Ray, and service-level metrics

stdapi.ai emits structured JSON logs for every request, stream, and background task, and integrates with OpenTelemetry (OTel) for traces and metrics. This guide shows how to enable observability, read the logs, and correlate signals across systems.

Quick Start¶

Set these environment variables, then restart the service (see the Configuration Guide for details):

# Set minimum log level (optional, defaults to "info")
# Options: info, warning, error, critical, disabled
export LOG_LEVEL=warning

# Enable OpenTelemetry tracing
export OTEL_ENABLED=true
export OTEL_SERVICE_NAME=stdapi
# 0.0–1.0 (10% example)
export OTEL_SAMPLE_RATE=0.1

# Include request/response payloads in logs (for debugging ONLY)
export LOG_REQUEST_PARAMS=true

# Log client IP addresses (requires ENABLE_PROXY_HEADERS for real client IPs)
export LOG_CLIENT_IP=true
export ENABLE_PROXY_HEADERS=true  # When behind ALB/CloudFront

Event Types¶

stdapi.ai emits five kinds of JSON events (one per line):

Common Fields¶

Each event shares core fields and may add type‑specific ones.

Usage Metrics Fields¶

Usage is reported as a nested usage list on request / request_stream events. Each entry represents billed quantities for a specific (service, model, operation, region, tier, routing, context) combination. Zero/empty fields are omitted.

Entry Structure¶

CloudWatch Metrics (EMF)¶

When CLOUDWATCH_METRICS=true is enabled, usage is emitted as CloudWatch Embedded Metric Format (EMF) lines to stdout. These are automatically extracted as CloudWatch Metrics on ECS with no extra IAM/API calls.

Configuration¶

EMF Line Structure¶

Each billed usage entry generates one EMF JSON line:

{
  "_aws": {
    "Timestamp": 1748716800000,
    "CloudWatchMetrics": [{
      "Namespace": "stdapi",
      "Dimensions": [["Model"]],
      "Metrics": [
        {"Name": "InputTokens", "Unit": "Count"},
        {"Name": "OutputTokens", "Unit": "Count"}
      ]
    }]
  },
  "Model": "anthropic.claude-sonnet-5",
  "operation": "/v1/chat/completions",
  "InputTokens": 1500,
  "OutputTokens": 450
}

Metric Details¶

• Single dimension: Model — provides low cardinality for cost control. A second [Model, Currency] dimension set is added for the Cost metric — see Cost Management for why cost needs its own dimension set.
• operation: Included as a queryable field (not a dimension)
• Metric units: Count for all token/image/character counts, Seconds for audio duration

fields @timestamp, Model, InputTokens, OutputTokens
| filter _aws.CloudWatchMetrics is not null
| sort @timestamp desc
| limit 20

Cost Tracking and Attribution¶

Real-time cost computation (COST_TRACKING), the price catalog and its accuracy caveats, the Cost EMF metric, and AWS-side cost attribution now live on their own page.

Cost Management

Correlating Logs and Traces¶

• Group events by id to reconstruct a full request lifecycle (request → stream(s) → background).
• The x-request-id response header exposes the same value so external systems can propagate correlation.
• With OTel enabled, a root span named like POST /v1/... is created and carries attributes: http.method, http.url, http.user_agent, request.id, server.id, http.status_code, and duration_ms.

Reading the Logs¶

• High latency: Inspect execution_time_ms on the request event. If the response was streamed, also sum request_stream durations. Combine with OTel spans to locate downstream delays (model provider, S3, etc.).
• Errors: Look for level=critical and error_detail (formatted exceptions). With OTel, the span is marked error with attributes error=true and error.message.
• Payload issues: Temporarily enable LOG_REQUEST_PARAMS=true to validate requests/responses, then disable.
• Client identification: client_user_agent and optional request_user_id / request_org_id help tie requests to users.
• Routing confirmation: model_id and voice_id confirm which provider/model/voice handled the request.

Controlling Log Verbosity¶

The LOG_LEVEL environment variable controls which log events are written to STDOUT. Set it to filter out lower-severity events. For detailed configuration options, see the Logging Level section in the Configuration Guide.

• info (default): All events are logged (info, warning, error, critical)
• warning: Only warnings and higher severity (warning, error, critical) - recommended for production
• error: Only errors and critical events
• critical: Only critical events
• disabled: No log output (not recommended)

# Production example: reduce log volume while maintaining visibility
export LOG_LEVEL=warning

OpenTelemetry Integration¶

When OTEL_ENABLED=true:

• A span is created per request and for streaming/background blocks.
• Spans carry request.id and server.id for correlation.
• 4xx/5xx status_code marks the span with an error status.
• Sampling is controlled via OTEL_SAMPLE_RATE.

For exporters and advanced setup, rely on standard OTel environment variables supported by your exporter/backend.

Example Events¶

Example — Request with payload logging enabled

{
  "type": "request",
  "level": "info",
  "date": "2025-01-01T12:00:00Z",
  "server_id": "stdapi-1",
  "id": "a1b2c3d4",
  "method": "POST",
  "path": "/v1/chat/completions",
  "status_code": 200,
  "model_id": "anthropic.claude-sonnet-5",
  "execution_time_ms": 842,
  "request_params": {"messages": [{"role": "user", "content": "..."}]},
  "request_response": {"id": "cmpl_...", "choices": [...], "usage": {...}}
}

Example — Streaming segment (SSE/audio)

{
  "type": "request_stream",
  "level": "info",
  "date": "2025-01-01T12:00:01Z",
  "server_id": "stdapi-1",
  "id": "a1b2c3d4",
  "execution_time_ms": 1234
}

Example — Background work correlated to a request

{
  "type": "background",
  "level": "info",
  "date": "2025-01-01T12:00:02Z",
  "server_id": "stdapi-1",
  "id": "a1b2c3d4",
  "event": "image-upload-s3",
  "execution_time_ms": 97
}

Example — Error with captured details

{
  "type": "request",
  "level": "critical",
  "date": "2025-01-01T12:00:05Z",
  "server_id": "stdapi-1",
  "id": "e9f0a1b2",
  "method": "POST",
  "path": "/v1/images/edits",
  "status_code": 500,
  "error_detail": ["Traceback (most recent call last): ..."],
  "execution_time_ms": 12
}

CloudWatch Logs Insights Queries¶

These examples assume JSON logs in CloudWatch Logs (default with ECS awslogs/awsfirelens). Adjust the log group and time range.

1) Follow a specific request across request/stream/background¶

fields @timestamp, type, level, path, event, status_code, execution_time_ms
| filter id = "<paste-request-id>"
| sort @timestamp asc

2) Find recent errors with context¶

fields @timestamp, level, type, path, status_code, id, error_detail
| filter level in ["error", "critical"]
| sort @timestamp desc
| limit 100

3) High-latency endpoints (P95/P99)¶

fields path, execution_time_ms
| filter type = "request" and ispresent(execution_time_ms)
| stats pct(execution_time_ms, 95) as p95_ms, pct(execution_time_ms, 99) as p99_ms, avg(execution_time_ms) as avg_ms by path
| sort p95_ms desc

AWS Service-Level Logs and Metrics¶

Beyond stdapi.ai logs and OTel traces, use AWS-native signals from the underlying AI services to validate provider behavior, monitor throttling/latency, and audit access. Enable only what you need: some options can capture content and increase costs. For details, refer to the official AWS documentation.

• CloudWatch Metrics: Throughput, latency, throttling, and error rates per service/region.
• CloudTrail: Control-plane auditing of API calls (who did what, when, from where).
• Content/Invocation logging: Optional features that may record inputs/outputs. Use with caution and encryption/retention controls.
• Correlation: Service logs won't include stdapi.ai's x-request-id. Correlate by time window, region, model/voice/job identifiers, and volume. Use stdapi.ai's model_id, voice_id, and execution_time_ms to narrow windows.
• Amazon Bedrock Invocation logging (optional): Export invocation metadata and, if enabled, content to CloudWatch Logs/S3/Firehose. Treat prompts/completions as sensitive; manage retention and KMS.

AWS Service Correlation Metadata¶

stdapi.ai attaches correlation identifiers to outgoing service calls, allowing you to trace a stdapi.ai request back to the corresponding AWS service invocation.

Coverage and how to use it varies by service:

• Bedrock — synchronous inference (every chat, embedding, image and audio route served by bedrock-runtime): identifiers are embedded in the invocation request — both the Converse and the InvokeModel families, streaming included — and appear in Bedrock invocation log records when model invocation logging is enabled. Filter by requestMetadata.stdapi-ai\.request_id in CloudWatch Logs Insights to join a Bedrock record with its stdapi.ai event.
• Bedrock — asynchronous batch jobs: identifiers are attached as resource tags on the job, visible in the AWS Console and searchable via the CLI/API.
• Bedrock Mantle: the Mantle endpoint accepts no request metadata. Attribute those requests with a Project/Workspace instead.
• Transcribe — audio transcription: identifiers are attached as job tags. stdapi.ai deletes completed transcription jobs automatically, so tags are only available while the job is still running.

Correlation also works in the other direction — from a stdapi.ai log event to the exact AWS API calls it made:

• aws_requests (on request, request_stream and background events): one entry per downstream AWS API call, with service, operation, request_id and — for failed calls — the AWS error code. The request_id value is the AWS-side request ID (ResponseMetadata.RequestId / x-amzn-requestid header): the identifier AWS Support asks for, and the requestID field of CloudTrail records. Each event keeps at most the 50 most recent calls.
• Edge headers (on the request event, when present): amzn_trace_id (X-Amzn-Trace-Id, injected by ALB and used by X-Ray), apigw_request_id (x-amz-apigw-id, API Gateway) and cloudfront_request_id (X-Amz-Cf-Id, CloudFront) tie the gateway's id to the edge service's own access logs. They are recorded as-is for correlation and never replace the gateway-generated request ID.

Infrastructure Observability¶

Troubleshooting Checklist¶

• No logs visible: Ensure you are reading container STDOUT. On ECS/Kubernetes, verify the log driver and retention.
• Missing request_params: Confirm LOG_REQUEST_PARAMS=true and restart after changing environment variables.
• No traces: Verify OTEL_ENABLED=true and that exporters are configured and reachable.
• Correlation missed: Ensure clients read and propagate x-request-id for multi‑hop requests.

Next Steps¶