Cost Management¶

Running stdapi.ai bills into three independent buckets:

AI service usage dominates any normal deployment. The gateway runs on a small ARM64 container, so its infrastructure and license form a low fixed floor that model spend passes quickly. Read the AI-cost sections first; Gateway Cost matters mainly at low traffic or under a hard cost constraint.

stdapi.ai adds no markup: model usage is billed to you by AWS at the same rate as calling Bedrock directly.

Knowing the Price Before You Call¶

The Model Pricing API (GET /model_pricing) exposes the AWS unit prices stdapi.ai resolved for each model, from the same published-price catalog it uses to estimate costs. Use it for cost-aware model selection — comparing candidates before routing traffic, rather than discovering the rate afterwards in the logs. Without a deployment, the Models page publishes the same rates per region and tier.

Marketplace-Billed vs AWS-Billed Models¶

A separate AWS Marketplace line on your bill for Bedrock usage is expected, not an error. AWS states it plainly:

Why It Matters: AWS Credits¶

This is the practical consequence, and it is easy to get wrong when planning a budget:

The same applies to committed-spend agreements: check whether your EDP or Private Pricing Agreement counts Marketplace spend before assuming a discount applies.

Which Models Are Which¶

Some models on Amazon Bedrock are sold as third-party AWS Marketplace listings; the rest bill as ordinary Amazon Bedrock usage. Open-weight models are generally in the second group. Which side a model sits on is the provider's and AWS's decision, it changes without notice, and a model can move after launch — so check your own account rather than trusting any list, this page included.

What stdapi.ai Does About It¶

• Both paths are priced. Cost tracking ingests Marketplace listings and native Bedrock rows alike, so a request log entry carries a cost regardless of how AWS bills it.
• Subscriptions are handled automatically. AWS creates the Marketplace subscription on first invocation; stdapi.ai keeps a not-yet-subscribed listing in the catalogue so that call can happen, when AWS_BEDROCK_MARKETPLACE_AUTO_SUBSCRIBE is enabled (the default), and hides it otherwise. It requires aws-marketplace:Subscribe and aws-marketplace:ViewSubscriptions — see IAM Permissions. Without them, the first call to a third-party model fails with AccessDeniedException.
• Cost tracking does not separate the two lines. Request logs report what a call cost, not which AWS invoice section it lands on. Use Cost Explorer, grouped by service, to see the Marketplace split.

Cost Tracking (Estimated from Published AWS Prices)¶

Cost tracking is opt-in and off by default. When COST_TRACKING=true is enabled, stdapi.ai estimates the cost of every request from AWS's published prices — an estimate computed at request time, not your billed amount read back from AWS. Costs are attributed to the actual region where each request was served.

How It Works¶

1. Price Catalog: At startup, stdapi.ai fetches the AWS Price List API for all configured regions and services in a background task, then caches in memory. Server readiness never waits on it: requests served before the load completes simply record usage without a cost. Failed loads are retried with exponential backoff (1–15 min), and each attempt's outcome is logged as a background event named price_catalog_load
2. On-Demand Refresh: The catalog is refreshed whenever a newly available Bedrock model is discovered with no catalog entry yet — not on a proactive schedule. If that refresh's AWS call fails (Price List throttling, missing permissions), the failure never propagates: the errors are captured per region and service rather than raised, the triggering request completes normally, and the new model stays unpriced until a later refresh succeeds. The refresh itself emits no per-request diagnostic — the miss surfaces on first use through the unpriced-model error_detail described in step 5
3. Per-Request Computation: For each request, costs are computed by multiplying billed quantities by the resolved unit price
4. Built-in Defaults: A few models are not in the Price List API at all — the Stability AI Image Services, priced on the AWS pricing page, and OpenAI's hosted GPT models, priced on their Bedrock model cards. Those published rates ship built in, used only when AWS publishes no row for the model; COST_PRICE_OVERRIDES still takes precedence
5. Fallback on a Missing Price: Once the catalog has been fetched, if a specific model/dimension has no resolvable price in it, the cost field is omitted for that entry rather than blocking the request, and the request log carries a warning-level error_detail naming the model and unpriced dimensions (a hint to supply the missing price via COST_PRICE_OVERRIDES)

Configuration¶

Request Log Format¶

Each usage entry includes cost and currency when resolved:

{
  "service": "bedrock-runtime",
  "model": "anthropic.claude-sonnet-5",
  "operation": "/v1/chat/completions",
  "region": "us-east-1",
  "input_tokens": 1500,
  "output_tokens": 450,
  "cost": "0.004575",
  "currency": "USD"
}

Costs are plain-decimal strings rather than floats, so no precision is lost and no exponent or trailing zeros appear. The request-level total is also logged:

{
  "cost": {
    "USD": "0.012345"
  }
}

EMF Cost Metric¶

When CloudWatch metrics are enabled, a Cost metric (unit: None) is emitted under the ["Model", "Currency"] dimension set — a separate directive from the quantity metrics, which stay under ["Model"]. A single directive spanning both sets would also publish Cost bare-by-Model, silently summing across currencies. Query Cost with both Model and Currency dimensions:

{
  "_aws": {
    "CloudWatchMetrics": [
      {
        "Namespace": "stdapi",
        "Dimensions": [["Model"]],
        "Metrics": [{"Name": "InputTokens", "Unit": "Count"}]
      },
      {
        "Namespace": "stdapi",
        "Dimensions": [["Model", "Currency"]],
        "Metrics": [{"Name": "Cost", "Unit": "None"}]
      }
    ]
  },
  "Model": "anthropic.claude-sonnet-5",
  "Currency": "USD",
  "Cost": 0.004575,
  "InputTokens": 1500
}

Regional Price Fallback¶

Some models — mostly older/deprecated ones — aren't published in every region's Price List (e.g. priced in us-east-1 but not any EU region). A region with no price for a given model/dimension/tier borrows one from a nearby region instead of omitting the cost:

1. Prefers another region in the same geography (eu-west-3 tries other eu-* regions first)
2. Falls back to us-east-1, eu-west-1, or us-west-2 — the regions always fetched regardless of your configured Bedrock regions
3. Never crosses a partition boundary, since a copied price keeps its source currency: a region in a sovereign partition with no in-partition source is left unpriced rather than borrowing a differently-denominated price
4. If no source is available, the cost is omitted as usual

This is a substitute price, not the actual published price for that region.

Multi-Currency Support¶

stdapi.ai detects currency from the AWS partition: - Standard AWS: USD - AWS European Sovereign Cloud (EUSC): EUR - AWS US GovCloud: USD - AWS China: CNY

Costs are never summed across currencies — this safety behavior is always on, regardless of settings. Regional fallback never crosses a partition boundary, precisely so that a copied price cannot report the wrong currency: a sovereign-partition region with no in-partition source is left unpriced instead. The remaining way a single request's billed dimensions resolve to different currencies is the Price List publishing more than one currency for the same region and model. A usage entry that spans more than one currency reports a costs map (instead of cost/currency) with every currency's own amount:

{
  "service": "bedrock-runtime",
  "model": "anthropic.claude-sonnet-5",
  "input_tokens": 1500,
  "output_tokens": 450,
  "costs": {
    "EUR": "0.0021",
    "USD": "0.0028"
  }
}

When only one currency is involved, the entry's cost/currency reports that currency directly instead. The request-level cost total still aggregates cleanly per currency either way.

Routing-Tier Pricing¶

AWS prices some models differently per serving profile: the cross-region "global" routing profile is lower than the plain/regional rate (confirmed live: Claude Sonnet 4.5 input tokens at $3.30/M regional vs $3.00/M global; GPT-5.6 Luna, Sol and Terra are each about 9% cheaper on it — the ratio is regional = global × 1.1, so global is ~9% cheaper and regional ~10% dearer, which are the same figure read in opposite directions), while latency-optimized serving (requested via the X-Amzn-Bedrock-PerformanceConfig-Latency: optimized header) is higher. stdapi.ai tracks the profile that served each request ("routing": "global" or "latency") and prices it at the matching rate. A geographic (us./eu./apac.) profile is billed at the regional rate, which is what AWS charges for it.

Falling back is normal, not a gap: where AWS publishes no distinct global rate for a model, the request is priced at that model's regional rate rather than left uncosted. This covers models whose rates come from the AWS pricing page or a model card instead of the Price List API, too. Bedrock Mantle offers no cross-region inference at all — a Mantle request is always served in the Region it was sent to, and always billed at the In-Region rate.

Service Tiers as a Cost Lever¶

AWS prices each service tier differently: flex trades latency for a lower rate, priority does the opposite. Two ways to apply one without changing client code: DEFAULT_MODEL_SERVICE_TIERS pins a tier per model, and a model alias pins one per alias — publishing, say, a flex name for batch workloads and a priority name for interactive ones over the same model. Set AWS_BEDROCK_ALLOW_SERVICE_TIER_OVERRIDE to false to stop clients selecting another tier, and the cost profile you configured holds. Both settings and the override gate cover models served through the Bedrock Converse and InvokeModel APIs; a Bedrock Mantle-served model runs on the tier its own request names.

Batch Inference¶

Requests sent through the Batch API or the Message Batches API are billed by AWS at the model's published batch rate, roughly half its on-demand rate. stdapi.ai prices them at that rate, reported with "tier": "batch" in the usage entry, and falls back to half the standard rate when AWS publishes no batch rate for a model.

Usage is recorded once per batch, when the batch reaches a terminal state and its totals are read — not per request. The entry therefore appears on whichever read observed the end of the batch, which is a poll rather than the call that created it. A batch that is cancelled before its requests run records nothing.

Tenant AWS Credentials¶

A tenant that registered an AWS credential has its model invocations billed by AWS to its own account, not to the deployment's. The usage log stays complete but honest about who paid:

• Each such usage entry carries "billed_to": "tenant", with the token quantities recorded as usual.
• No cost is computed for those entries — pricing them would claim spend the deployment never incurred — and they are excluded from the request's cost totals and from the Cost CloudWatch metric.
• Everything else in the same request (Amazon Polly, Transcribe, S3, Knowledge Bases, moderation guardrail calls) remains the deployment's spend and is recorded and priced exactly as before.

To see what a credentialed tenant spends, read the tenant's AWS bill: Amazon Bedrock usage appears in the tenant account's Cost Explorer, attributable to the role sessions named stdapi-ai-tenant-<key id>.

Vector Stores¶

Indexing a file into a vector store costs one embedding call per passage, and a search costs one per query. Those calls are recorded and priced like any other embedding usage, against the model in VECTOR_STORE_EMBEDDING_MODEL — a large file is many passages, so the cost of an attach scales with the file, not with the request.

Indexing runs after the response is sent, so its usage is reported on a background log event of its own rather than on the request that started it. Correlate the two through the id field they share.

The storage and request charges of the vector storage itself are not recorded: like the bytes the Files API stores, they appear on your AWS bill and not in the usage log. Read them from AWS Cost Explorer, filtered on Amazon S3 Vectors — the service holding AWS_S3_VECTORS_BUCKET.

A deployment that sets AWS_SQS_VECTOR_STORE_QUEUE_URL adds Amazon SQS requests to that bill — a handful per attached file, plus one long-poll receive per idle server every 20 seconds. It adds no embedding cost: a job replayed after a server was replaced skips whatever already completed, so a file is never embedded twice.

A knowledge base store is billed differently, because the retrieval happens inside Amazon Bedrock and no embedding call of this server's is involved:

• A fully managed knowledge base charges a flat rate per retrieval call, with the document parsing, the embedding and the reranking included in it. Every query of a search is one retrieval, recorded as one search_units unit under the model amazon.bedrock-knowledge-base and priced from the rate on the Amazon Bedrock pricing page — the Price List API publishes no row for it, so that rate is built in and can be replaced through COST_PRICE_OVERRIDES.
• A knowledge base whose vector storage you provisioned yourself has no published per-retrieval rate: its search costs whatever its embedding model and its vector database charge, both billed outside this server's view. Nothing is recorded for it rather than a figure that would be invented.
• Attaching a file records nothing, on either kind. AWS publishes no per-document ingestion charge — a fully managed knowledge base includes parsing and embedding in its rates and bills its index storage monthly per GB of raw data, and the other kind bills the embedding model and the parser it is configured with.

Read all of these from AWS Cost Explorer, filtered on Amazon Bedrock.

Bedrock Marketplace Model Endpoints¶

A model served from a Bedrock Marketplace model endpoint you deployed to your own AWS account is billed on an entirely different model from everything else on this page: by the instance-hour, not the token.

Real on-demand SageMaker hosting rates for one ml.g6.xlarge instance, read from the AWS Price List API on 2026-08-26:

Check current rates for your own instance type and region on the SageMaker pricing page before deploying — these move, and they are not what stdapi.ai's own cost tracking reports for this path (see below).

What the gateway reports. Token counts for a Marketplace model endpoint are recorded in the request log and in /model_pricing under a service of their own — the same treatment as a customer-provisioned knowledge base's search, and for the same reason: no unit price is resolved and none is invented. AWS publishes no per-token rate for this path, because it does not bill per token. A request against a model endpoint therefore reports quantities with no cost, and raises no missing-price warning — that warning is reserved for a real gap in the price catalogue, not for a billing model this app has no rate to report. Read the actual charge from AWS Cost Explorer or the SageMaker hosting line of your bill.

Deploy, Test, Destroy¶

Because the meter runs from creation to deletion regardless of traffic, the only honest way to exercise this feature is deploy → test → delete, every time — create the endpoint, run whatever you need against it, then delete it immediately. Never leave one running.

A full cycle is roughly 10–15 minutes to deploy, a few minutes to use, a few minutes to delete — under $2 at the rates above:

aws bedrock create-marketplace-model-endpoint \
  --region <region> \
  --endpoint-name <name> \
  --model-source-identifier arn:aws:sagemaker:<region>:aws:hub-content/SageMakerPublicHub/Model/<listing>/<version> \
  --accept-eula \
  --endpoint-config "sageMaker={initialInstanceCount=1,instanceType=ml.g6.xlarge,executionRole=<role-arn>}"

# ... run your test, then always:
aws bedrock delete-marketplace-model-endpoint --region <region> --endpoint-arn <arn>

• --accept-eula accepts the listing's licence on your behalf — exactly why the gateway never calls either command itself: it only discovers an endpoint you already created, and never creates, updates or deletes one.
• The endpoint's execution role must be named to match *Sagemaker*ForBedrock* if you use AWS's documented iam:PassRole policy for this role.
• Deployment is gated by the SageMaker endpoint-usage quota per instance type, often 1 or 2 by default and adjustable through Service Quotas — request an increase before you need it, not after a deploy fails on it.

SageMaker AI Endpoints¶

A model served from an Amazon SageMaker AI endpoint you run is billed the same way as a Marketplace model endpoint: by the instance-hour, not the token. The same ml.g6.xlarge rates apply, and the same caveat about checking them on the SageMaker pricing page for your own instance type and Region.

What the gateway reports. Token counts are recorded in the request log and in /model_pricing under a service of their own, with no unit price resolved and none invented — AWS publishes no per-token rate for this path, because it does not bill per token. A request against a SageMaker AI endpoint therefore reports quantities with no cost, and raises no missing-price warning. Read the actual charge from AWS Cost Explorer, on the *-Host:ml.* usage type of the endpoint's Region.

Unlike a Marketplace model endpoint, this path can cost nothing at rest. An endpoint configured for scale to zero — inference components, a production variant with MinInstanceCount: 0, an Application Auto Scaling target with a minimum capacity of zero, and a step-scaling policy driven by a CloudWatch alarm on NoCapacityInvocationFailures — releases its instances when nothing is calling it, and provisions one again on the next request. Four things to budget for:

• The wake is a few minutes, absorbed by AWS_SAGEMAKER_WARMUP_TIMEOUT so the caller sees a slow request rather than an error.
• The scale-in tail costs more than the wake. AWS keeps the instance for a while after the last request, so one idle-to-idle cycle bills for noticeably longer than the traffic that caused it.
• The alarm outlives the traffic. NoCapacityInvocationFailures stays in ALARM for its evaluation window after the last rejected request, and the step-scaling policy keeps acting on it — so an endpoint scaled in by hand comes straight back up, and a burst of requests against a cold endpoint bills for 10 to 15 instance-minutes after the last caller has gone. Let it settle by itself rather than writing the copy count against the policy that owns it.
• The CloudWatch alarm that makes the wake happen at all lists at $0.10 per alarm-metric-month, inside CloudWatch's always-free allowance of ten alarms.

Everything else about an idle scale-to-zero endpoint is unbilled: no instances means no instance-hours and no EBS volume, the model, endpoint-configuration and inference-component objects carry no charge, Application Auto Scaling is free, and AWS/SageMaker metrics are not billed as custom metrics.

Long-Context Pricing¶

Some 1M-context-capable models are billed by AWS at a higher rate — roughly double for input-side tokens — once a call's prompt (input + cache read/write tokens) passes the boundary that model publishes. That boundary differs per model: Claude Sonnet 4 (via the context-1m anthropic-beta flag) switches at 200K tokens, while the OpenAI GPT-5.6 models bill their short-context rate all the way to 272K. stdapi.ai applies each model's own boundary, prices the whole call at the published long-context rate, and reports it with "context": "long" in the usage entry. When AWS publishes no long-context rate for a model, the standard rate is used as the best available estimate.

Built-in Tool Pricing¶

Amazon Nova's built-in grounding tool (web_search → nova_grounding, supported by Nova 2 and Nova Premier) is billed by AWS per grounding request, on top of token usage. stdapi.ai counts grounding invocations in the model response and reports them as grounding_requests, priced from the model's published per-request rate.

The Amazon Bedrock web search tool the OpenAI GPT-5.x family uses on /v1/responses is billed per query at one flat rate — the same for every model — in every Region where that tool is available. Because AWS publishes it without a model, it is reported on its own usage entry under the model amazon.bedrock-web-search, also as grounding_requests, beside the invocation's own token entry. A single turn may run several queries, and one tool call may issue more than one, so the count comes from the queries the response reports rather than from the number of tool calls; page reads have no published per-query rate and are not counted.

Image Pricing Granularity¶

Some image-generation models (currently: Amazon Titan Image Generator G1/V2 and Nova Canvas) are priced by AWS per resolution/quality combination, not a flat per-image rate. stdapi.ai automatically prices these per-image, based on the actual requested size and quality, falling back to a flat per-image rate for models where this isn't wired up yet (e.g. Stability). No configuration needed — this is purely additive precision with no accuracy trade-off.

Media Input Pricing¶

Multimodal embedding inputs are billed by AWS per media unit on top of (or instead of) tokens: per input image (with a distinct "document image" rate where offered) and per second of audio/video. stdapi.ai records image counts directly and audio/video durations from the AWS-reported segment timings of the asynchronous (segmented) processing path, reported as input_images/input_seconds with their *_by_spec breakdowns. Rerank queries are recorded as search_units (one per query).

Limits and behaviour to know¶

• Speech-modality tokens (speech-to-speech models, including Realtime API sessions): speech and text tokens are recorded in separate buckets and priced at their own published rates, so no modality is priced as the other. Where a model publishes no speech rate at all, its speech tokens fall back to the text rate, which underestimates speech-heavy calls. A Realtime session bills continuously in both directions for as long as it is open, not per request; usage is recorded per answer, matching each response.done event in the request log.
• Asynchronous (segmented) embeddings: AWS reports no token usage for this processing path (used automatically for large inputs), so segmented text embeddings report no token cost; audio/video durations are recovered from the AWS-reported segment timings and billed.
• Synchronous audio/video embedding inputs: media duration is only available from AWS on the segmented (asynchronous) processing path — small audio/video inputs processed synchronously report no duration and no per-second cost. No estimate is substituted (this app only reports AWS-confirmed real usage).
• Client disconnect during streaming: streamed chat responses still record their final usage after a disconnect. For streamed responses on other routes and for image generation jobs, AWS bills the input tokens and everything generated up to the cancellation, but no usage is recorded for that call. No estimate is substituted.
• Rerank queries with more than 100 documents: AWS bills one search unit per 100 documents; the document count isn't visible at recording time, so one unit per query is recorded.
• Reserved capacity pricing: if a request explicitly asks for AWS's Reserved Capacity service tier, its cost is computed at the standard on-demand rate instead — Reserved Capacity uses a separate monthly-commitment pricing model this app doesn't ingest. Avoid relying on this app's cost figures for Reserved Capacity workloads.
• Some very new or region-specific models may have no published price anywhere yet — AWS publishes pricing after model availability, sometimes with a delay.
• AWS GovCloud: the Price List API has no GovCloud endpoint, so catalog prices cannot be fetched there — usage is still recorded, a startup warning is emitted, and only COST_PRICE_OVERRIDES entries produce costs.

Override Map for Missing Models¶

Some models — recently released ones, or region-specific listings — may not appear yet in the AWS Price List API. Use COST_PRICE_OVERRIDES to fill gaps (Bedrock models only — Polly/Transcribe/Translate/Comprehend prices always come from the catalog):

export COST_PRICE_OVERRIDES='{"anthropic.claude-sonnet-5":{"input_tokens":0.000003,"output_tokens":0.000015}}'

Prices are per one unit (token, character, second) in your partition's currency.

IAM Permissions¶

Ensure your IAM role includes pricing read access (the Price List API serves identical data from its three commercial endpoints, and sovereign partitions such as EUSC host their own; the nearest one is selected automatically from your configured Bedrock regions):

{
  "Effect": "Allow",
  "Action": ["pricing:GetProducts"],
  "Resource": "*"
}

AWS Cost Attribution¶

Cost tracking prices each request as it happens. AWS-side attribution answers a different question — whose spend lands on the AWS bill, in Cost Explorer and CUR 2.0. The two are complementary: request logs for granular analysis, AWS attribution for invoicing and chargeback.

IAM Principal Attribution¶

Amazon Bedrock records the IAM identity behind every bedrock-runtime inference call and forwards it to Cost Explorer and CUR 2.0. Nothing to configure in stdapi.ai, and no change to client requests.

By default stdapi.ai calls AWS with a single execution role — the ECS task role — so all Bedrock spend is attributed to that one identity, the standard LLM-gateway pattern AWS documents. To break that total down, tag the execution role (for example team or cost-center), then activate those keys in the AWS Billing console under Cost allocation tags, filtering by type IAM principal. To split it per end user instead, see Per-User Attribution below.

Per-User Attribution¶

Every end user is attributed at the request level out of the box: the client-supplied identifier is recorded as stdapi-ai.user_id in the request log — alongside that request's computed cost — and forwarded to Bedrock as request metadata, so per-user spend can be aggregated from the logs.

To split the AWS bill itself per end user, give each one an identity of their own. Set AWS_BEDROCK_USER_ROLE_ARN to a role you create, and stdapi.ai opens a short-lived session of that role per end user and runs their model calls under it. AWS then reports each user separately in Cost Explorer and CUR 2.0, from the invoice rather than from the logs.

Usage API Query Cost¶

The Organization Usage and Costs API answers from the metrics this deployment publishes to Amazon CloudWatch, and CloudWatch bills both halves of that: the metric series it stores every month, and every metric a query reads. This is the one feature on this page whose own cost can rival a small model workload, so it is off by default and priced here rather than discovered on an invoice.

Rates below are us-east-1, read from the AWS Price List API (AmazonCloudWatch) on 2026-08-26 and cross-checked against the CloudWatch pricing page. They differ by region and they move — price your own before enabling.

What a Query Costs¶

On a deployment serving ~79 models — fewer than the Models page lists, because a deployment only serves the models its regions and account access give it — one group_by=model query against /v1/organization/usage/completions reads roughly 108 series:

That figure is per polling client: two dashboards on the same schedule cost twice as much, and a shorter poll interval scales it linearly until the response cache absorbs it.

Count your own instead of scaling this one, because series follow traffic rather than catalogue size. A model contributes one series per endpoint operation it actually served (chat.completions, completions, responses, messages and realtime all feed the completions endpoint) for each usage quantity that carried a value — InputTokens, OutputTokens and Requests on any call, CachedTokens and CacheWriteTokens only where prompt caching applies. Multiply the model/operation pairs your deployment really serves by the quantities each reports, then by the GetMetricData rate above; a model in the catalogue that nobody calls costs nothing here.

What Enabling It Costs at Rest¶

Turning USAGE_API on also publishes the usage metrics under an additional Model+Operation dimension set, so the endpoints can answer per-model and per-endpoint. Those extra series are stored metrics, billed monthly whether or not a single query is ever made — roughly $120–$180/month on a 79-model deployment, scaling with the number of models actually used.

The Four Protections¶

Each is a setting, and each default is chosen to keep the bill predictable:

Lower USAGE_API_MAX_METRICS and raise USAGE_API_CACHE_TTL on a large catalogue; the defaults are a ceiling, not a target.

Gateway Cost¶

Secondary for most deployments — read this section if you run under a hard cost constraint or at low traffic, where the gateway's fixed floor is a visible share of the bill.

Infrastructure¶

The gateway is a single small container. The Terraform module defaults to ARM64, 0.25 vCPU and 512 MiB — the smallest Fargate size — which keeps this bucket cheap, though it is still a floor you pay whether or not any request arrives.

That default sizes the task for text generation and embeddings. Serving audio, video or inline file inputs holds bytes in memory and needs memory raised to 1024 or beyond, which raises this floor — see the sizing note.

The Terraform module provisions:

Cost Tiers at a Glance¶

Both infrastructure and license cost scale with how many containers run and for how long — there's no single "deployment cost," just a wide range between a minimal setup and a full production one. Three representative configurations:

None of these include AI service usage (Bedrock, Polly, Transcribe, …), which is billed separately by AWS at cost — see the buckets table at the top of this page.

Keeping It Low¶

• Stop the service outside business hours. autoscaling_schedule_stop and autoscaling_schedule_start take cron expressions, so the cluster can run weekdays only and cost nothing at night — the single largest saving for an internal workload.

  autoscaling_schedule_stop  = "cron(0 19 ? * MON-FRI *)"
  autoscaling_schedule_start = "cron(0 8 ? * MON-FRI *)"
  

• Reuse existing network infrastructure. Passing subnet_ids and security_group_id deploys into your VPC and creates no additional NAT gateways or load balancers — see Integration with Existing Infrastructure. On a small deployment these are often larger than the gateway itself.

• Scale to your real floor. autoscaling_min_capacity sets how many containers run at idle. Because the Commercial License is billed per container-hour, this number drives both infrastructure and license cost.
• Use Fargate Spot and trim log retention, as the Cost-Optimized Deployment example does.
• Trim observability volume. Raising LOG_LEVEL and disabling request/response payload logging cut CloudWatch ingestion — see Controlling Log Verbosity. Note that EMF metric lines are not affected by LOG_LEVEL: they are written to stdout on every request whenever CLOUDWATCH_METRICS is on, so disabling that setting is the only way to remove them.

Model List Sharing¶

MODEL_CACHE_SHARED is the one optional feature whose own AWS cost is worth stating rather than rounding away. Left off, or with no AWS_DYNAMODB_TABLE configured, it costs nothing: no table is created, no request is made. The table alone, created but unused, is also $0 — DynamoDB on-demand has no idle charge, this table holds kilobytes, and its storage sits inside the 25 GB always-free allowance.

Enabled, it is not free. What it costs is the published list itself:

• Writes dominate. Each refresh writes the compressed model list, and DynamoDB bills a write in 1 KB units. At the default MODEL_CACHE_SECONDS=900 that is 96 refreshes a day for the whole fleet, whoever performs them.
• Reads round to nothing. Each server reads the list once per interval — a few thousand read units a day — plus one small conditional write to claim the refresh.
• Traffic does not change any of it. The table is touched when the list expires, never per request, so this is a fixed monthly figure rather than one that scales with usage.

On DynamoDB on-demand pricing in us-east-1 ($0.625 per million write request units, $0.125 per million read request units, both higher in some regions), 96 refreshes a day of a list in the low hundreds of kilobytes comes to roughly $0.60–$2 a month for the whole fleet, plus a few cents per server for the reads.

License¶

stdapi.ai is dual-licensed: the free AGPL-3.0-or-later license costs nothing but requires sharing your modifications, while the Commercial License — billed per container-hour via AWS Marketplace, no per-request or per-token component — is what production deployments typically run. See Licensing for the full pricing table and trial/private-offer terms.

Next Steps¶