2026-09-05 · 11 min
API Rate Limiting in Node.js: Token Bucket vs Leaky Bucket (Production Patterns)
Your API hit 429 responses at 3 AM because a single client hammered /api/export 400 times in under a minute. No malicious intent — just a misconfigured cron job from a partner’s server somewhere near Pasar Modern BSD Serpong. Without API rate limiting in place, that traffic saturated your database connection pool and caused latency spikes for every other user.
This happens more often than people admit. Rate limiting is treated as something you add later, after the first incident. This article covers how to implement it properly in Node.js from the start, using token bucket and leaky bucket algorithms backed by Redis, with the kind of specificity that makes the difference between a rate limiter that works in development and one that holds up in production.
Why algorithm choice matters
Two requests at 100 ms apart behave identically in a fixed-window counter — they both count toward the same bucket. A client can send 999 requests at 11:59:59 PM and another 1000 at 12:00:00 AM and never be throttled, despite sending nearly 2000 requests in two seconds. Fixed-window counters are easy to implement and wrong in ways that are invisible until they matter.
Token bucket and leaky bucket solve the boundary problem differently. Understanding the mechanics determines which one fits your use case.
Token Bucket
A bucket has a maximum capacity of N tokens. Every interval, tokens are added up to the cap. Each request consumes one token. If the bucket is empty, the request is rejected with 429.
The critical property: a client can burst up to N requests immediately as long as tokens have accumulated. A bucket with capacity 100 and a refill rate of 10 tokens/second allows a burst of 100 requests at once, then settles to 10 requests/second steady state.
Here is a Redis-backed token bucket using Lua to keep the check-and-decrement atomic:
// lib/rate-limit/token-bucket.ts
import { createClient } from "redis";
const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();
// Lua script: atomic check + decrement, returns [allowed, tokensRemaining, resetAt]
const TOKEN_BUCKET_LUA = `
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refillRate = tonumber(ARGV[2]) -- tokens per second
local now = tonumber(ARGV[3]) -- current timestamp in ms
local bucket = redis.call('HMGET', key, 'tokens', 'lastRefill')
local tokens = tonumber(bucket[1])
local lastRefill = tonumber(bucket[2])
if tokens == nil then
-- First request: initialize bucket
tokens = capacity
lastRefill = now
end
-- Refill tokens based on elapsed time
local elapsed = (now - lastRefill) / 1000 -- convert to seconds
local refilled = math.floor(elapsed * refillRate)
tokens = math.min(capacity, tokens + refilled)
lastRefill = now
if tokens >= 1 then
tokens = tokens - 1
redis.call('HMSET', key, 'tokens', tokens, 'lastRefill', lastRefill)
redis.call('EXPIRE', key, math.ceil(capacity / refillRate) + 60)
return {1, tokens, -1}
else
local waitSeconds = (1 - tokens) / refillRate
redis.call('HMSET', key, 'tokens', tokens, 'lastRefill', lastRefill)
return {0, 0, math.ceil(waitSeconds)}
end
`;
// Load the script once at startup — EVALSHA sends only the hash on subsequent calls
const scriptSha = await redis.scriptLoad(TOKEN_BUCKET_LUA);
export async function checkTokenBucket(
key: string,
capacity: number,
refillRate: number // tokens per second
): Promise<{ allowed: boolean; remaining: number; retryAfter: number }> {
const now = Date.now();
const result = (await redis.evalSha(scriptSha, {
keys: [key],
arguments: [String(capacity), String(refillRate), String(now)],
})) as [number, number, number];
return {
allowed: result[0] === 1,
remaining: result[1],
retryAfter: result[2],
};
}
The reason to use EVALSHA over EVAL on every request: Redis caches the compiled Lua bytecode by SHA. The script travels over the wire once at startup. Every subsequent call sends only the 40-character hash. At high throughput this is measurable.
Leaky Bucket
Leaky bucket processes requests at a fixed rate regardless of when they arrive. Incoming requests are added to a queue; the queue drains at a constant interval. If the queue is full, the request is rejected.
The output is uniform. No matter how fast a client sends requests, the downstream service sees at most drainRate requests per second.
// lib/rate-limit/leaky-bucket.ts
// Uses Redis sorted set: score = scheduled process time, member = request ID
const LEAKY_BUCKET_LUA = `
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local drainRateMs = tonumber(ARGV[2]) -- ms between each request
local now = tonumber(ARGV[3])
local requestId = ARGV[4]
-- Remove expired entries (already processed)
redis.call('ZREMRANGEBYSCORE', key, '-inf', now)
local queueSize = redis.call('ZCARD', key)
if queueSize >= capacity then
return {0, queueSize, -1}
end
-- Schedule this request: after all queued requests drain
local lastScheduled = redis.call('ZRANGE', key, -1, -1, 'WITHSCORES')
local scheduleAt
if #lastScheduled == 0 then
scheduleAt = now
else
scheduleAt = math.max(now, tonumber(lastScheduled[2]) + drainRateMs)
end
redis.call('ZADD', key, scheduleAt, requestId)
redis.call('PEXPIRE', key, capacity * drainRateMs + 5000)
local waitMs = scheduleAt - now
return {1, capacity - queueSize - 1, math.ceil(waitMs / 1000)}
`;
const leakySha = await redis.scriptLoad(LEAKY_BUCKET_LUA);
export async function checkLeakyBucket(
key: string,
capacity: number,
requestsPerSecond: number
): Promise<{ allowed: boolean; queueRemaining: number; waitSeconds: number }> {
const drainRateMs = Math.floor(1000 / requestsPerSecond);
const requestId = `${Date.now()}-${Math.random()}`;
const result = (await redis.evalSha(leakySha, {
keys: [key],
arguments: [String(capacity), String(drainRateMs), String(Date.now()), requestId],
})) as [number, number, number];
return {
allowed: result[0] === 1,
queueRemaining: result[1],
waitSeconds: result[2],
};
}
Express middleware wiring
Both algorithms plug into Express the same way. The key design decision: use composite keys so you can apply different limits to different routes without duplicating middleware instances.
// middleware/rate-limit.ts
import { Request, Response, NextFunction } from "express";
import { checkTokenBucket } from "../lib/rate-limit/token-bucket";
interface RateLimitConfig {
capacity: number;
refillRate: number; // tokens per second
keyFn: (req: Request) => string;
}
export function tokenBucketMiddleware(config: RateLimitConfig) {
return async (req: Request, res: Response, next: NextFunction) => {
const key = config.keyFn(req);
const { allowed, remaining, retryAfter } = await checkTokenBucket(
key,
config.capacity,
config.refillRate
);
// Always set headers — even on allowed requests
res.setHeader("X-RateLimit-Limit", config.capacity);
res.setHeader("X-RateLimit-Remaining", remaining);
if (!allowed) {
res.setHeader("Retry-After", retryAfter);
res.status(429).json({
error: "rate_limit_exceeded",
message: `Too many requests. Retry after ${retryAfter} seconds.`,
retryAfter,
});
return;
}
next();
};
}
Applying different limits per route:
// routes/api.ts
import { Router } from "express";
import { tokenBucketMiddleware } from "../middleware/rate-limit";
const router = Router();
// Broad per-user limit: 200 req/min
const userLimit = tokenBucketMiddleware({
capacity: 200,
refillRate: 200 / 60,
keyFn: (req) => `ratelimit:user:${req.user?.id ?? req.ip}:global`,
});
// Tight limit on expensive endpoint: 5 req/min
const exportLimit = tokenBucketMiddleware({
capacity: 5,
refillRate: 5 / 60,
keyFn: (req) => `ratelimit:user:${req.user?.id ?? req.ip}:export`,
});
// Apply both: export limit checked first, short-circuits before global limit
router.get("/export", exportLimit, userLimit, handleExport);
router.get("/data", userLimit, handleData);
Evaluate the tighter limit first. If export is rejected at 5/min, there is no reason to spend another Redis round-trip checking the 200/min global bucket.
Memory footprint at scale
One detail that bites teams who Redis-back everything without thinking: each active key consumes memory. A token bucket key stores two hash fields (tokens, lastRefill). A leaky bucket key stores a sorted set with up to capacity members.
At 10,000 active users:
- Token bucket: roughly 10,000 × ~150 bytes = ~1.5 MB. Negligible.
- Leaky bucket with capacity 50: 10,000 × (50 × avg member size ~80 bytes) = ~40 MB. Still fine, but worth monitoring.
Set EXPIRE on every write. The Lua scripts above do this. Without expiry, keys from one-time clients accumulate indefinitely — that is a slow Redis memory leak that shows up months later and is annoying to diagnose.
When to use which
Token bucket for user-facing APIs, webhooks, and mobile clients. Users expect responsive behavior; occasional bursts (opening the app after a long idle, batch-syncing) are legitimate. Token bucket allows that burst and throttles gracefully after.
Leaky bucket when protecting a downstream service that charges per call or has documented rate limits itself — payment gateways, SMS providers, third-party APIs. You do not want your clients’ burst behavior to cause you to hit the provider’s limit. Leaky bucket absorbs the burst on your side and releases it at a rate the downstream service can handle.
One configuration I have used in production: leaky bucket on outbound calls to a payment provider (10 req/s, queue depth 30), token bucket on the inbound API endpoint that triggers those calls (capacity 100, refill 20/s). The inbound side bursts are queued and smoothed before they reach the provider.
Graceful degradation when Redis is unavailable
Do not let a Redis failure take down your API. Wrap the rate limit check in a try-catch and define a fallback policy:
try {
const result = await checkTokenBucket(key, capacity, refillRate);
if (!result.allowed) {
return res.status(429).json({ error: "rate_limit_exceeded" });
}
} catch (err) {
// Redis is down: log and fail open (allow the request)
// Alternatively, fail closed (reject all) for sensitive endpoints
logger.error("Rate limit Redis error", { err, key });
// Allow request through — Redis outage should not cause outage
}
next();
Fail open (allow requests) for standard endpoints, fail closed (reject all with 503) for endpoints that have significant cost or abuse potential. The choice depends on which failure mode is more acceptable: degraded protection or degraded availability.
The case against library defaults
Most teams reach for express-rate-limit with rate-limit-redis store and ship the default fixed-window algorithm. This works well enough for basic abuse prevention. It does not work when a client needs to be throttled smoothly, when you have multiple backend instances, or when request cost varies.
If your use case is simple — one server, basic bot protection, no burst requirements — fixed-window with express-rate-limit is fine and adds no implementation burden. If you have multiple Node.js instances behind a load balancer, or you need to protect a paid downstream service, or you want to offer different limits to different client tiers, implementing token bucket or leaky bucket in Redis yourself is the right call. The Lua scripts above are around 30 lines each and will not surprise you six months later.
Rate limiting is one of those things where the generic solution and the correct solution are often not the same thing.
Ditulis oleh Reza Pradipta