Skip to content

Message Batches API

Run a large set of Messages API requests asynchronously, at a lower price than the synchronous API, through the Anthropic Message Batches API shape.

The requests are sent inline, the batch runs without a connection held open, and its results are streamed back as JSONL — exactly the Anthropic workflow, so the official Anthropic SDKs work by changing the base URL.

Why Choose the Message Batches API?

  • Lower Price per Token
    Batched requests are billed at the published batch rate, well below the on-demand rate for the same model.

  • Drop-in Anthropic Compatibility
    client.messages.batches.create(...) and .results(...) work unchanged.

  • Several Models, One Batch
    Each request names its own model, as upstream allows; the batch reports a single aggregate state.

  • Private AWS Backend
    Requests and results are stored in your own S3 buckets — no traffic to third-party endpoints.

Available Endpoints

Endpoint Method What It Does MCP Tool
/anthropic/v1/messages/batches POST Create a batch from inline requests anthropic_message_batch
/anthropic/v1/messages/batches GET List batches, newest first anthropic_message_batch_list
/anthropic/v1/messages/batches/{id} GET Retrieve a batch and its counters anthropic_message_batch_get
/anthropic/v1/messages/batches/{id}/results GET Stream the results as JSONL anthropic_message_batch_results
/anthropic/v1/messages/batches/{id}/cancel POST Cancel a batch that is still processing anthropic_message_batch_cancel
/anthropic/v1/messages/batches/{id} DELETE Delete a batch that has ended anthropic_message_batch_delete

Feature Compatibility

Feature Status Notes
Creation
requests[].custom_id Up to 64 characters, unique within the batch
requests[].params Same parameters as Messages
Several models in one batch Up to 8, each needing the 100-request minimum
tools / tool_choice Refused when the batch is created — tool use is not available in a batch
Structured output schema Refused when the batch is created
stream A batch has nothing to stream to
cache_control Accepted and ignored — a batch reads and writes no prompt cache, and the request is answered without one
Lifecycle
Retrieve / poll in_progress → canceling → ended
Results (JSONL) Streamed; available once processing_status is ended
Cancel Requests that already produced a Message keep it, the ones that never ran are reported canceled; cancelling twice, or cancelling a batch that has ended, changes nothing
Delete Only once the batch has ended — cancel it first, as upstream requires
List batches Newest first, with before_id / after_id cursors
archived_at Results stay readable until the batch is deleted

Legend:

  • Supported — Fully compatible with Anthropic API
  • Partial — Supported with limitations
  • Unsupported — Not available in this implementation

results_url and Reverse Proxies

results_url is an absolute URL on the address the request came in on, so client.messages.batches.results(...) works with no extra configuration and a client fetching it outside the SDK gets a URL it can dial as-is. Behind a reverse proxy it names the proxy's own origin, taken from the Host and X-Forwarded-Proto headers — enable ENABLE_PROXY_HEADERS so the forwarded scheme is trusted, or a TLS-terminating proxy yields an http:// URL.

Content Guardrails and Batches

A request that a guardrail would apply to is refused rather than run unguarded. Send those requests without batching.

Prompt Caching and Batches

Batched requests neither read nor write a prompt cache, on any model. A request carrying cache_control is still accepted and answered — the hint is dropped rather than the request — so a result reports no cache_read_input_tokens and no cache_creation_input_tokens. Nothing is lost by leaving it in: batched requests are already billed at the batch rate, and the cache discount was never available at that rate.

Model Support

Any chat model available for batch inference in your configured Amazon Bedrock regions can be used — the same identifiers as Messages. To shortlist them, call search_models with route=anthropic_message&batch=true; each entry also carries a batch field.

The shortlist is a hint, not a rule

batch is advertised on a best-effort basis and never used to reject a request. A model it does not advertise — or says nothing about — may still run a batch, so submit the batch rather than ruling the model out; the answer you get back is the authoritative one.

A model that cannot serve batched requests is refused when the batch is created, naming the model; no sibling job is left running. A model this deployment normally serves through another Amazon Bedrock endpoint is batched under the identifier the batch endpoint knows it by, so it needs nothing from you.

Workflow

1. Create the batch

from anthropic import Anthropic

client = Anthropic(base_url="https://your-host/anthropic", api_key="...")

batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": f"req-{index}",
            "params": {
                "model": "amazon.nova-micro-v1:0",
                "max_tokens": 256,
                "messages": [{"role": "user", "content": text}],
            },
        }
        for index, text in enumerate(documents)
    ]
)

Example request (curl):

curl -X POST "https://your-host/anthropic/v1/messages/batches" \
  -H "x-api-key: $API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {
        "custom_id": "req-1",
        "params": {
          "model": "amazon.nova-micro-v1:0",
          "max_tokens": 256,
          "messages": [{"role": "user", "content": "Summarize: ..."}]
        }
      }
    ]
  }'

The samples above are abridged: a real batch needs at least 100 requests for each model it names.

Example response:

{
  "id": "msgbatch_06fvfg3lbdqarbad8kbo55g0sg5h3s4a",
  "type": "message_batch",
  "processing_status": "in_progress",
  "request_counts": {"processing": 100, "succeeded": 0, "errored": 0, "canceled": 0, "expired": 0},
  "created_at": "2026-08-12T20:53:33Z",
  "expires_at": "2026-08-13T20:53:33Z"
}

2. Poll until processing ends

batch = client.messages.batches.retrieve(batch.id)
print(batch.processing_status, batch.request_counts)

succeeded and errored move as the batch runs, so they can drive a progress bar. canceled and expired are known only once processing_status is ended.

3. Read the results

for entry in client.messages.batches.results(batch.id):
    if entry.result.type == "succeeded":
        print(entry.custom_id, entry.result.message.content[0].text)

Each line pairs a custom_id with its outcome:

{"custom_id": "req-1", "result": {"type": "succeeded", "message": {"id": "msg_req-1", "type": "message", "role": "assistant", "content": [{"type": "text", "text": "..."}], "stop_reason": "end_turn", "usage": {"input_tokens": 22, "output_tokens": 9}}}}
{"custom_id": "req-2", "result": {"type": "errored", "error": {"type": "error", "error": {"type": "invalid_request_error", "message": "..."}}}}

Results Are Not in Request Order

Result lines may come back in any order, as upstream also warns. Match a result to its request with custom_id, never with the line number.

Limits

Limit Value
Minimum requests per model 100 (default quota)
Maximum requests per batch 100,000
Distinct models per batch 8
Processing window 24 hours from creation

A batch below the minimum, or over the model cap, is refused when it is created and the message names the shortfall — a batch naming several models must reach the minimum for each of them.

The 100-request minimum is a quota default

100 is the default of the Amazon Bedrock quota Minimum number of records per batch inference job, which is set per model and adjustable for some of them — see Amazon Bedrock quotas. The gateway checks against that default, not against your account's own value, so a raised quota is enforced by Amazon Bedrock rather than here — a model given 150 requests clears this check and is then refused by the backend — and a lowered one is not usable: fewer than 100 requests for a model is still refused here.

Prerequisites

The Message Batches API is disabled until the deployment declares an AWS IAM service role that Amazon Bedrock assumes to read the requests and write the results:

The permissions the role and the server need are listed in IAM Permissions. While the role is unset, every batch endpoint answers 529.

Billing

Batched requests are billed at the published batch rate for the model, roughly half the on-demand rate. Usage is recorded once, when the batch ends. See Cost Management.

See Also