Rate Limiting APIs: Token Buckets, Global Limits, Quotas, and 429 Responses

Rate Limiting APIs: Token Buckets, Global Limits, Quotas, and 429 Responses

Rate limiting protects shared capacity by controlling how quickly one caller can consume a resource. It is useful for abuse prevention, tenant fairness, downstream protection, and cost control.

Pick the Right Identity

Limits can be keyed by:

IP address
user
API key
tenant
route/resource

IP-only limits are often insufficient behind NATs and easy to distribute around. For authenticated APIs, user/key/tenant limits are usually more meaningful.

Token Bucket

A token bucket allows a controlled average rate while permitting short bursts.

Other algorithms such as fixed windows, sliding windows, and leaky buckets have different accuracy/memory/burst trade-offs. Choose based on behavior, not algorithm popularity.

Local vs Global Limits

A limit enforced independently on every API replica can multiply as replicas increase.

Use local limits for fast per-instance protection and a coordinated/global limit when the policy must apply across the whole fleet. Envoy explicitly supports both patterns.

Limit More Than Requests per Second

OWASP's API resource-consumption guidance recommends bounding inputs and expensive operations too.

Examples:

• maximum upload/body size
• pagination/result limits
• batch size
• concurrent requests
• OTP/password-reset attempts
• AI/token/spend budgets
• export/report frequency

Ten extremely expensive requests can be worse than a thousand cheap ones.

Return Useful 429 Responses

Use HTTP 429 Too Many Requests when the caller exceeds a request policy. Retry-After can tell clients when another attempt is appropriate.

Clients should use backoff rather than immediately retrying every 429 and creating more load.

Protect Downstream Capacity

A public API may scale faster than its database or third-party provider. Rate/concurrency limits should cap work before the constrained dependency becomes saturated.

Final Takeaway

Effective rate limiting is resource-aware and identity-aware. Combine burst control, global tenant/key quotas, concurrency limits, bounded inputs, and clear 429 semantics so traffic spikes or abusive clients cannot turn into database saturation or runaway cost.

References

• OWASP API4: Unrestricted Resource Consumption
• MDN: 429 Too Many Requests
• Envoy Global Rate Limiting