It’s 2:47 AM on a Tuesday when PagerDuty fires. The alert title says “PostgreSQL - connection pool exhausted.” The body says active_connections: 100/100. You open the dashboard. Traffic looks normal - around 200 requests per minute across the whole app - except for one endpoint. POST /api/signup is fielding 10,000 requests per minute. From a single IP range in eastern Europe.

The requests are syntactically valid. Real-looking names. Email addresses with plausible domains. Passwords that pass your complexity rules. Each one triggers a SELECT for username uniqueness, another SELECT for email uniqueness, a bcrypt.hash that burns 100ms of CPU, and then an INSERT. One hundred database connections occupied. All of them processing garbage. Real users trying to sign up get a 502 Bad Gateway because the connection pool has nothing left to give.

You block the IP. Requests shift to a different one. You block that. Five more appear. By 3:15 AM you are manually playing whack-a-mole against a rotating proxy network, your only weapon being the database that is slowly drowning under the load. You are losing.

The post-mortem will note - without irony - that the fix took eleven minutes to deploy. The monitoring failure took three weeks to discover. The absence of rate limiting had existed for four years.

This is the rate limiting problem. Five algorithms exist to solve it. Most teams pick the wrong one.

Why This Happens

Open APIs are, by default, unconditional. Your server has no concept of “this IP is sending too much traffic” unless you explicitly encode that rule somewhere. HTTP is stateless. Each request arrives as if it were the first.

The failure chain is mechanical:

Bot sends 10,000 signup requests/min
  → API server accepts all (no guard)
    → Each request: SELECT + SELECT + bcrypt(100ms) + INSERT
      → Connection pool: 100 slots fill instantly
        → Legitimate requests queue behind bot traffic
          → Queue depth grows: 50, 500, 5,000
            → Requests timeout before connection granted
              → HTTP 502 / 503 for real users
                → Business: incident, refunds, churn

The deeper problem is that bot traffic looks like real traffic at the HTTP layer. You cannot distinguish intent from a TCP packet. What you can do is enforce a policy: no single identity - IP, user ID, API key, or device fingerprint - gets unlimited access to a resource. The mechanism for enforcing that policy is rate limiting.

The Naive Solution (and Where It Breaks)

The first thing most engineers reach for is fixed window counter rate limiting. Track how many requests an IP has made in the current time window. Reject requests that exceed the limit.

The implementation is simple:

def check_rate_limit(ip: str, window_seconds: int = 60, max_requests: int = 100) -> bool:
    # Key rotates every window_seconds
    key = f"ratelimit:{ip}:{int(time.time() // window_seconds)}"
    count = redis.incr(key)
    if count == 1:
        redis.expire(key, window_seconds)
    return count <= max_requests

At small scale this works fine:

100 users, 5 req/min each: peak count = 5  →  well under limit  →  no problem

At real bot scale, it breaks in a specific way: