Deciding who gets through, how often, and with what fairness.
What does a rate limiter have to be fair about: user, API key, IP, region, or endpoint?
Before reading, choose the key, window, and failure behavior for a public API.
Rate limiting interleaves gateway behavior, distributed counters, consistency, latency, and abuse control.
A rate limiter protects systems from overload and abuse by deciding whether the next request should pass or be rejected. The core idea is small, but the details matter: who is being limited, how the rule works, where the limiter sits, and whether the state must work across many servers.
The clean first version is usually simple: define the limiting key, define the rule, keep current usage in a fast store, then allow or reject quickly. That is enough to make the chapter practical without getting lost in low-value implementation detail.
The most useful simplification is this: before choosing an algorithm, define who you are limiting.
Interviewers like rate limiter because it tests whether the candidate can identify the right limiting key, pick a reasonable algorithm, talk about distributed state, and discuss correctness versus simplicity without getting abstract.
The stronger answer starts with the rule and the identity, then chooses the mechanism.
A rate limiter is just a Redis counter.
A rate limiter is a policy plus algorithm plus distributed state problem, with fairness and failure behavior explicitly chosen.
State the limit key, algorithm, storage location, and fallback when the limiter store is slow.
| Limiter choice | Good when | Weak when | Interview line |
|---|---|---|---|
| Local per-instance limiter | The system is small and rough protection is enough. | Traffic is spread across many servers and fairness matters across the fleet. | A local limiter is simple, but it is only an approximation once traffic hits multiple servers. |
| Shared distributed limiter Default | Many servers handle requests and you want a more consistent global view. | The added network hop and shared state are not justified yet. | A shared limiter makes sense when distributed fairness matters more than absolute simplicity. |
| Fixed window counter | Simplicity matters most and boundary effects are acceptable. | Burstiness at window edges makes the limit too easy to game. | Fixed window is simple, but it is the least fair around boundaries. |
| Token bucket Default | Short bursts are acceptable but sustained abuse should be capped. | You need a different fairness model and burst allowance is undesirable. | Token bucket is a good fit when I want to allow bursts while controlling the sustained rate. |
| Sliding window style approach | Fairer smoothing matters more than minimal implementation complexity. | Simplicity is more important than tighter accuracy. | A sliding model is more accurate, but I would only pay that cost if fairness really matters. |
This is where the chapter becomes interesting. The limiter itself is simple. The hard part is making many doors act like one bouncer.
This is the useful product-fairness test. The algorithm matters, but the real question is whose pain you are willing to create.
Do not jump straight from reading to a full answer. First see the shape, then complete part of it, then answer alone.
I would say: "I will limit by API key with a token bucket so short bursts are allowed but sustained abuse is blocked."
For a public API, compare fixed window and token bucket in one sentence each.
Answer the practice prompt and include what happens when the counter store is unavailable.
Before moving on, turn recognition into production. Close the model answer, answer from memory, then retry one small slice.
Say the chapter's core idea without looking. Then name one related idea from an earlier chapter.
Change one constraint in the practice prompt and answer again in half the time.
Use the rubric to pick one dimension below 3, then retry only that dimension.
The limiting key shapes everything else.
A limiter that sits too late is less useful than it sounds.
Distributed consistency always comes with a price.