Rate limiting: token bucket vs leaky bucket

When traffic spikes or an automated script hammers your endpoints, a database without rate limiting will collapse within seconds.

Rate limiters enforce fair usage policies and protect backend services. In system design, two primary algorithms dominate: Token Bucket and Leaky Bucket.

Token Bucket:  Tokens added at constant rate  ──>  Requests consume tokens (Bursty)
Leaky Bucket:  Requests fill queue bucket     ──>  Processed at steady constant rate (Smooth)

1. Token Bucket: Handles bursts of traffic

A bucket holds a maximum capacity of tokens (e.g. 10 tokens).

A background ticker adds tokens at a fixed rate (e.g. 2 tokens per second).
When a request arrives, it attempts to draw 1 token from the bucket.
If tokens > 0, the request is processed and token count decrements.
If the bucket is empty, the request is rejected with 429 Too Many Requests.

[ + Token / sec ] ──> ┌──────────┐
                      │ ● ● ● ●  │ (Max 10)
                      └───┬──────┘
                          │ (Takes 1 token)
                      Incoming request ──> PASS or 429

Why it’s popular: Token bucket allows short, legitimate bursts of traffic (e.g. loading an initial dashboard that fires 8 API calls in parallel) while enforcing an average rate over time. Used by AWS, Stripe, and GitHub APIs.

2. Leaky Bucket: Smooths output rate

In a leaky bucket, incoming requests enter a FIFO queue of a fixed capacity.

The bucket “leaks” (processes requests) at a strictly constant rate (e.g. 5 requests per second).
If requests arrive faster than the leak rate, the buffer queue fills up.
Once the buffer is full, any incoming requests spill over and return 429 Too Many Requests.

Why it’s useful: It provides completely smooth downstream throughput, making it ideal for systems with strict downstream throughput constraints (like third-party payment gateways or SMS providers).

How rate limits communicate over HTTP

Standard HTTP headers keep the client informed:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1727092800
Content-Type: application/json

{
  "error": "rate_limit_exceeded",
  "message": "Too many requests. Please try again in 30 seconds."
}

Algorithm Comparison

Feature	Token Bucket	Leaky Bucket	Sliding Window Log
Traffic shape	Allows bursts up to capacity	Strictly smoothed constant rate	Highly accurate, no edge boundary issues
Memory footprint	Tiny (counter + timestamp)	Small (FIFO queue buffer)	Larger (stores timestamps of every request)
Best suited for	Public Web APIs (Stripe, GitHub)	E-commerce checkouts, message dispatchers	Strict financial APIs
Implementation	Redis `INCR` / Lua script	Redis streams or Celery queue	Redis Sorted Sets (`ZADD`, `ZREMRANGEBYSCORE`)