Bedrock ThrottlingException: add a region, add its quota
TL;DR¶
Amazon Bedrock quota is per region and per model. If you are seeing ThrottlingException, you are usually not at the limit of what AWS will sell you. You are at the limit of the one region you deployed in. Adding a second region brings that region's own quota with it, and it needs no conversation with your account team.
Cross-region inference profiles are the first thing to try, and they draw on a separate quota again. After that, three limits catch people: a stream cannot fail over once it has started, an asynchronous invocation stays in the region that accepted it, and more regions is not automatically more throughput.
It is usually not a capacity problem¶
ThrottlingException reads like a wall. Most of the time it is a topology detail.
Every AWS region maintains its own independent Bedrock quota. Not a shared pool with regional views of it: separate numbers, tracked separately. Nothing you are granted in eu-west-1 has any effect on us-east-1, and a quota increase approved for one is invisible to the other.
The quota is also per model. anthropic.claude-* and amazon.nova-* in the same region have separate tokens-per-minute quotas, and separate requests-per-minute quotas where the model has one at all, because AWS enforces RPM for some models and not others. Burning through one leaves the other untouched.
So there are two dimensions, and the practical consequence is that capacity planning which treats "our Bedrock quota" as a single number is planning against a number that does not exist. The thing you actually have is a matrix.
What a second region buys¶
The matrix is also the lever. Pointing traffic at a second region adds that region's quota to what you can reach, without asking anyone.
It is the cheapest throughput increase available on Bedrock. A quota increase request is a support ticket with a review and a wait. Provisioned throughput is a commitment with a bill attached. Reaching a second region is an IAM policy and a first call: serverless models enable themselves on first invocation given the AWS Marketplace permissions, and the region's default quota is usually available from that call. Check the applied value in Service Quotas before planning on it: new accounts can carry reduced applied quotas in every region, and only a support case fixes that. If it is an opt-in region, the account has to be opted in first.
What it is not is a multiplier. Three regions does not mean three times the throughput, for reasons in the section on limits below, and anyone selling it to you that way is rounding off the part that matters.
Start with cross-region inference profiles¶
Before building anything, use what AWS ships. A cross-region inference profile is a single model identifier that AWS routes across several regions for you: no infrastructure, no config to maintain, and it solves the common case.
It also has a quota of its own. Bedrock counts Cross-Region InvokeModel tokens per minute separately from On-demand InvokeModel tokens per minute, per model and per region, and global profiles get a third bucket again. So calling through a profile is not spending the allowance your direct calls spend, which is worth establishing before you conclude you are out of capacity. That bucket is counted in the region you call from, so the same profile called from a second source region draws on a second bucket. The identifier is how you tell the two apart: anthropic.claude-... is the in-region on-demand path, and us. or eu. in front of it is the cross-region profile. They do not share an allowance.
Three things to check before planning around it.
A geography-scoped profile has a fixed destination list. AWS states that a profile tied to a geography such as US or EU will never change the regions it routes to. It is a set you select, not a set you compose, so if your allowed regions do not line up with a published geography, the profile is not expressing your constraint.
A global profile is the opposite, and that is the catch. It routes to supported commercial regions worldwide and its destination list grows as AWS adds them. If you have a residency position, this is the tier that breaks it: permitting it means permitting aws:RequestedRegion to be unspecified in your policies, which is a decision to take deliberately rather than to discover.
Not every model carries one. Embedding models are the usual surprise. Check the model you actually want rather than the model in the example.
If a geography fits your constraint and your model has a profile, stop here. You do not need anything else, and adding a gateway to do what an inference profile already does is overbuying.
When a profile is not enough, and why it gets postponed¶
The remaining case is real: your permitted regions are not a geography, or the model has no profile, or you want to spread across profiles and on-demand capacity at once. That is where the work appears.
You declare the same model once per region. Each entry carries its own tokens-per-minute and requests-per-minute limits, so the config grows as the product of models and regions rather than the sum. LiteLLM's router works exactly this way: each model is declared once per region, and TPM and RPM go on each entry if you want its usage-based routing rather than the default shuffle.
You maintain an availability list that AWS keeps invalidating. Model availability by region changes on AWS's schedule, not yours. Every launch, every regional expansion and every deprecation edits your config, and nothing tells you it has happened except a failure.
You write retry logic in the application, or in a proxy you configure and operate. Doing it in the application is the awkward one, because that layer has the least idea which regions are currently healthy. It knows the call failed. It does not know whether the next region is any better, and by the time it finds out it has spent the user's latency budget discovering it.
The three limits that catch people¶
1. A stream cannot fail over once it has started. Retry works before the stream opens. Once bytes are flowing the region is locked, and nothing in front of Bedrock changes that. So a mid-stream failure surfaces to the user, and if your product is a chat UI, this is the failure mode you will actually see.
2. An asynchronous invocation stays where it landed. A StartAsyncInvoke job, the API behind video and other long-running generation, selects its region when it starts and nothing in front of Bedrock moves it afterwards. Batch inference is the exception, not the rule: submit it through a cross-region inference profile and AWS spreads the compute across the profile's regions itself, against batch's own separate quota. Only the job record and its S3 input and output stay in the region you submitted from.
3. More regions is not automatically more throughput. This is the one most often skipped in "multi-region for resilience" write-ups. Quota is per region and per model, and the model you want may exist in a handful of regions rather than all of them. Adding a region that does not carry your model adds nothing at all. Check availability first and plan capacity second, not the other way round.
The constraint that comes first¶
Which regions you are allowed to enable is not a capacity decision. It is usually a data-residency decision, made by someone who has never seen a ThrottlingException and who is not going to be persuaded by one.
Get the allowed list before you design the routing. It is the outer constraint and everything else works inside it. Encoding it as an IAM or SCP deny rather than as team discipline is also what lets you show an auditor a policy instead of an intention.
Where I put it, and where it stops¶
I build stdapi.ai, an AI gateway you run in your own AWS account, so my answer is that this belongs in the gateway: you enable the regions you are allowed to use, requests route across them, and eligible failures retry on the next one. Throttling, quota exhaustion, a temporary regional outage, or a region that cannot currently serve the model. The application keeps sending one base URL and one model name.
Three places it stops, and all three are properties of the problem rather than of the implementation.
A read timeout does not fail over. Every other retryable error on a request without S3 inputs escalates to the next region immediately, but on a read timeout the model has already been invoked and AWS bills it whatever the client does. Failing over there would pay a second region for the same generation rather than recover it, so the request returns 503 instead. That is a cost limit rather than a capability one, and it is the kind worth stating plainly.
Each candidate region is tried at most once per request. A region that just failed is under the backoff its own failure recorded, and a second error there would only deepen it. So the number of attempts is bounded by how many regions can serve the model, or by the configured retry cap, whichever runs out first: you cannot buy more attempts than you have regions.
A request carrying S3 inputs is pinned to one region. An S3 reference is resolved for the region that holds the data and cannot be replayed against another, so those requests keep botocore's in-region retries and never move region at all.
None of those is unique to this gateway. Any honest router hits all three.
What to check first¶
- Which regions is this model actually available in?
- Which of those are you permitted to use, and is that written as a policy?
- Does a cross-region inference profile exist for it, and does its geography match your permitted list?
- Are you reading the on-demand quota or the cross-region quota when you conclude you are throttled?
- What share of your traffic is streaming, given that streaming cannot fail over mid-response?
- Is anything on the critical path an asynchronous invocation, given that those do not move region once started?
How are you handling Bedrock throttling today? Cross-region inference profiles, your own retry layer, or provisioned throughput?