Picture a hotel’s electronic keycard system. The front desk programs a card for a specific guest with a specific set of doors it will open: room 412, the gym, the pool, never the penthouse and never the housekeeping closets. If that guest loses the card in the lobby, the front desk cannot wait until checkout to deal with it. The lost card has to stop opening every single door in the building within seconds, not just the door the front desk happens to be standing next to, and every other guest’s card has to keep working exactly as before. An API key management system is that hotel’s access-control backend rebuilt for software: it issues credentials scoped to exactly what a client should be allowed to do, and when one leaks or gets revoked, that decision has to reach every enforcement point almost instantly.

Consider a platform issuing keys to roughly 5 million active third-party integrations, fielding 60,000 requests a second on average and spiking past 200,000 requests a second during a partner’s own peak traffic, spread across a gateway fleet of 150-plus edge nodes in multiple regions. Every one of those requests carries an API key in a header, and every one has to be authenticated, checked against the scopes the key was actually granted, and counted against that key’s own rate limit, all before the request is allowed to touch the origin service. When a key is compromised or a customer rotates off a leaked credential, the platform has committed to shutting that key down everywhere, on every one of those 150 gateway nodes, within one second.

The naive approach is a synchronous database lookup on every request: hash the incoming key, SELECT the row, check status = 'active', check the scopes column, and let the request through. That fails the moment you do the math. At 200,000 requests a second, a single authoritative datastore taking even a single row lookup per request becomes the busiest system in the entire architecture, and it sits directly in the latency budget of every request the platform serves, including the ones that have nothing to do with key management. Push authorization checks onto a synchronous, centralized dependency and you have turned an internal bookkeeping system into the platform’s most fragile single point of failure.

The second failure mode shows up once you try to fix the first one with a plain cache. Cache the key’s status at the gateway with a generous TTL, say five minutes, and the hot path gets fast again, but now a revoked key keeps working at any gateway node holding a stale cache entry for up to five minutes after the platform believes it has been shut down. Shrink the TTL to make revocation faster and you are back to hammering the database, just on a tighter cycle. Neither a synchronous check nor a long-TTL cache alone satisfies both constraints at once. We need to solve for three things simultaneously: authorizing a scoped request off a cache that lives entirely on the gateway’s own hot path, propagating a revocation to every gateway node inside a hard one-second budget without a synchronous read on every request, and recording detailed per-key usage for billing and analytics without that recording ever being on the critical path of the response.

Requirements and Constraints

Functional Requirements

• Issue API keys scoped to specific resources and actions (orders:read, orders:write, webhooks:manage), with a key never usable outside its granted scopes
• Verify a key and its scopes on every incoming request and reject anything outside its granted scope set with a 403
• Enforce a distinct rate limit per key (requests per second and a burst allowance), configurable per plan tier, independent of any other key’s traffic
• Revoke a key on demand (manual action, automated leak detection, customer offboarding) and guarantee that revocation is honored by every gateway node within 1 second
• Support key rotation with a grace period so a client can adopt a new secret before the old one stops working, with zero downtime for the integration
• Record usage per key (request count, endpoint, status code, bytes transferred) for billing and for a per-key usage dashboard
• Expose an admin API to create, list, scope, rotate, and revoke keys, and a read API for customers to see their own key’s usage

Non-Functional Requirements

• Hot path latency: authorization and rate-limit check add no more than 2ms at p99 to a request’s total latency, since this sits in front of every API call the platform serves
• Throughput: sustain 200,000 requests/second at peak across the gateway fleet, 60,000/second sustained average, each individually authenticated and rate-limited
• Scale: 5 million active keys, roughly 150 gateway nodes across 3+ regions, around 300 million usage events per day
• Revocation SLA: a revoked key must stop being accepted on every gateway node within 1 second of the revocation being issued, with no manual intervention
• Availability: 99.99% for the authorization path; the gateway must fail toward a safe, bounded default (serve from last-known-good cache, never block all traffic) if the control plane is unreachable
• Durability: key records and revocation history are never lost; usage analytics can tolerate seconds of delay but not silent data loss
• Consistency: strong consistency for the authoritative key record in the control plane, bounded eventual consistency (sub-1s) for gateway-side enforcement

Constraints and Assumptions

• We assume TLS terminates before the gateway and focus on application-layer key validation, not transport security
• We do not design the customer-facing dashboard UI in depth, only the APIs and data model behind key management and usage reporting
• OAuth2 token flows (short-lived bearer tokens, refresh tokens) are out of scope; this is a long-lived, directly-presented API key model, the pattern used by Stripe, GitHub, and most B2B platform APIs
• A single global control plane with regionally replicated read paths is assumed; we discuss cross-region propagation only where the revocation SLA depends on it
• We do not enumerate every possible scope taxonomy for every API surface; the goal is the mechanism for scoped authorization, not a fixed list of scopes for any one product

High-Level Architecture

The system has six major components: the Key Issuance Service, the Gateway Auth Middleware (with its local cache), the Per-Key Rate Limiter, the Revocation Propagation layer, the Usage Event Pipeline, and the Key Rotation Workflow.

The Key Issuance Service is the control plane: it generates keys, hashes them for storage, assigns scopes and rate-limit tiers, and is the single source of truth for a key’s current status. The Gateway Auth Middleware runs on every edge node in front of the origin services; it extracts a key from the request, checks it against a local, short-TTL cache before ever considering a call back to the control plane, and enforces scopes. The Per-Key Rate Limiter is a sharded counter store, typically Redis, that the gateway calls after a key passes authentication, tracking request counts per key independently of every other key. Revocation Propagation is the mechanism that pushes a “this key is no longer valid” event out to all 150-plus gateway nodes the instant an admin or an automated detector revokes a key. The Usage Event Pipeline receives a lightweight, fire-and-forget event from the gateway after every request and rolls it up asynchronously into per-key usage analytics. The Key Rotation Workflow lets a key holder generate a new secret while the old one keeps working for a bounded overlap window, so rotating a credential never requires a synchronized cutover.

A request flows like this: a client calls the API with a key in the Authorization header. The gateway hashes the key, checks its local in-memory cache for that key’s metadata (status, scopes, rate-limit config), and on a cache hit, which is the overwhelming majority of requests, never talks to any other service to authenticate the request. It checks the requested scope against the key’s granted scopes, calls the rate limiter with the key’s identifier, and if both checks pass, forwards the request to the origin and fires an async usage event. On a cache miss, the gateway fetches the key’s metadata from the control plane once, populates its local cache, and proceeds. Separately, whenever a key’s status changes, the Key Issuance Service publishes a revocation event that every gateway node subscribes to, invalidating that key in every local cache within the propagation SLA regardless of each node’s own cache TTL.

Component Deep Dives

The Key Issuance Service

This component’s job is to be the single authoritative source for what a key is allowed to do and whether it is still valid, and to never hand out or store a key’s raw secret after the moment it is created.

Key hashing for storage means the platform stores a one-way hash of the secret, never the secret itself, so that a database breach does not hand an attacker a list of usable credentials. A generated key looks like sk_live_7hQ3mZ..., a static prefix plus a long random secret; the service stores the prefix (for display and lookup) and an HMAC-SHA256 hash of the full secret, keyed with a server-side pepper that never lives in the database.

# Key generation and hashing at issuance time
# Demonstrates: cryptographically random secret, HMAC hash with a server-side pepper,
# constant-time verification so timing differences never leak information about the hash
import hashlib
import hmac
import secrets

PEPPER = b"loaded-from-a-secrets-manager-not-the-database"  # never stored alongside key_hash

def generate_api_key(env: str = "live") -> tuple[str, str, bytes]:
    raw_secret = secrets.token_urlsafe(32)          # ~256 bits of entropy
    prefix = f"sk_{env}_{raw_secret[:8]}"
    full_key = f"sk_{env}_{raw_secret}"
    key_hash = hmac.new(PEPPER, full_key.encode(), hashlib.sha256).digest()
    return full_key, prefix, key_hash               # full_key is shown to the caller exactly once

def verify_api_key(candidate_key: str, stored_hash: bytes) -> bool:
    candidate_hash = hmac.new(PEPPER, candidate_key.encode(), hashlib.sha256).digest()
    # compare_digest runs in constant time regardless of where the strings first differ,
    # which matters because a timing side channel would let an attacker guess a hash byte by byte
    return hmac.compare_digest(candidate_hash, stored_hash)

Scope-based authorization is assigned at issuance and stored as a compact set on the key record: a scope like orders:write names a resource and an action, and a key only ever gets the scopes its owner explicitly grants it. The prefix is deliberately not secret; it is what shows up in a “last used” dashboard and what a leak-detection scanner matches against, while the secret half never round-trips back to the server in cleartext after creation.

Think of the prefix and secret split like a hotel room number printed on the outside of a keycard sleeve next to the card itself: the room number is not sensitive, it just tells the desk clerk which card this is at a glance, while the actual magnetic strip is what a locked door checks. If you simplified this component to storing keys in plaintext, or to a reversible encryption scheme instead of a one-way hash, a single database dump would hand an attacker every live credential on the platform at once, which is exactly the class of breach key_hash with HMAC and a pepper is designed to survive.