ORIGINAL TECHNICAL PRACTICE
Choose a rate-limit contract
How would you design a rate limiter for an API used by both individuals and batch clients?
Written for independent practice. The model answer presents one approach; clarify constraints and explain alternatives.
A REASONED APPROACH
Model answer.
Define identity, time scale, burst allowance and the resource being protected. A token bucket supports bounded bursts with a controlled refill rate; a concurrency limit protects a different dimension.
Work through it
- Select keys such as authenticated account and endpoint class. Consider shared-IP users, anonymous traffic and costly endpoints without relying only on a client-supplied identifier.
- Enforce the chosen algorithm with atomic state updates. In a distributed deployment, define whether a regional approximation is acceptable or a shared limit is required.
- Return an appropriate limit response and useful retry timing when known. Test concurrent requests, clock assumptions, key expiry and the limiter’s own storage outage.
Tradeoffs to discuss
- A centralized limiter can enforce a tighter global quota while adding latency and a dependency. Local limits are more resilient but can allow a larger aggregate burst.
Common pitfalls
- A fixed window can permit a double burst near a boundary. Rate limits alone do not prevent too many slow requests from remaining active.