Picture a bank account shared by five siblings, each holding their own debit card, all drawing against the exact same balance. If two siblings check the balance at the same instant, both see $200, and each walks up to a different ATM three blocks apart and withdraws $150, the bank has no way of knowing about the overdraft until both withdrawals have already been dispensed. The account was never actually short of money at either individual moment of checking, it only became short once both withdrawals landed, and by then the cash is already in someone’s pocket.

Now replace the five siblings with fifty regional warehouses, replace the two ATMs with a checkout button that gets clicked by thousands of shoppers a second, and replace the shared bank balance with the last unit of a doorbuster item during a flash sale. That is the oversell problem: a piece of physical stock that exists in exactly one place can be shown as “available” to two different shoppers at once if the system checking availability and the system recording a sale are not the same atomic operation. Selling something you do not have is not a rounding error, it is a promise the business cannot keep, and the fix a customer support team offers afterward (a refund, a discount, an apology email) costs far more than the sale was worth.

The naive fix, decrement the count the moment payment succeeds, sounds safe but creates its own failure: a shopper adds the last unit to their cart, gets distracted by a slow card entry form, and thirty seconds later a second shopper across the country sees the item as “in stock,” because nothing was held for the first shopper while they typed. Decrementing too late lets two people believe they bought the same unit. Decrementing too early, the moment an item is added to a cart with no expiration, creates the opposite failure: an abandoned cart holds inventory hostage forever, and a warehouse can show “sold out” on an item that is, in physical reality, sitting untouched on a shelf. Both mistakes come from treating “browsing,” “checking out,” and “paying” as if they happen instantly and atomically, when in reality a real checkout session can take anywhere from three seconds to several minutes, and across that window somebody else is always trying to buy the same thing.

We need to solve for three things simultaneously: an atomic counter that fifty warehouses can update concurrently without ever going negative, a time-bound soft hold that reserves stock for exactly as long as a checkout session needs and no longer, and a routing layer that can satisfy an order from whichever of the fifty warehouses actually has the stock, even when that means splitting the order across more than one.

Requirements and Constraints

Functional Requirements

• Reserve a requested quantity of a SKU against warehouse stock the moment a shopper begins checkout, attaching a TTL (default 15 minutes) to that reservation
• Commit a reservation into a permanent sale the instant a payment webhook confirms success, converting the soft hold into a durable decrement of on-hand stock
• Release a reservation automatically, without any client action, the moment its TTL expires or the shopper explicitly abandons checkout
• Route a reservation request across whichever warehouse or combination of warehouses in the network can fulfill it, splitting into multiple shipments when no single warehouse holds the full requested quantity
• Support both an optimistic, retry-based concurrency strategy and a pessimistic, lock-based strategy, and pick between them per SKU based on how contended that SKU currently is
• Emit a sold-out signal to the product catalog and search layer the instant a SKU’s network-wide available count reaches zero, and a back-in-stock signal once it recovers
• Guarantee that every reserve, commit, and release API call is idempotent, so a client’s network retry never creates a duplicate hold or a duplicate release

Non-Functional Requirements

• Network scale: 50 regional warehouses, roughly 2,000,000 distinct SKUs, on the order of 500,000,000 total units on hand across the network
• Peak reservation throughput: sustained 15,000 reservation requests/sec in normal operation, bursting to 80,000 requests/sec during flash sales and major promotional windows
• Latency: reserve p99 under 50ms, commit and release p99 under 100ms
• Correctness guarantee: the available count for any warehouse/SKU pair must never go negative under any level of concurrency; this is a harder requirement than latency or throughput
• TTL: default hold duration of 15 minutes, configurable per campaign, with expired holds released back to the available pool within 5 seconds of expiry
• Availability: the reserve path, being directly on the revenue path, targets 99.99% uptime; fulfillment routing is allowed to degrade to a single-warehouse fallback rather than take the whole reserve path down
• Consistency scope: a reservation is scoped to a single request’s set of warehouses; there is no requirement to keep two independent reservations for the same SKU strictly ordered relative to each other beyond the correctness guarantee above

Constraints and Assumptions

• A separate catalog and pricing system owns product metadata; this system only owns quantity counts and reservation state, not price or description
• The actual physical pick, pack, and ship process is owned by warehouse operations systems; this system’s job ends once a reservation is committed into a sale
• The payment gateway’s webhook is treated as the source of truth for “did the customer actually pay,” we do not design payment processing itself
• Warehouses report physical stock counts (receiving, cycle counts, returns) into this system through a separate replenishment feed, which we treat as an upstream input, not something this design owns
• Cross-region inventory sharing (routing an order from a customer in one country to a warehouse in another) is out of scope for this design; all 50 warehouses are assumed to sit inside one logical fulfillment region

High-Level Architecture

The system has six major components: the Reservation Gateway, the Distributed Inventory Counter Store, the Fulfillment Routing Engine, the Reservation Ledger, the TTL Expiry Sweeper, and the Sold-Out Signal Bus.

The Reservation Gateway is the front door: it validates a reservation request, deduplicates retries using an idempotency key, and hands the request to the routing layer. The Fulfillment Routing Engine decides which warehouse, or which combination of warehouses, should actually supply the requested quantity, scoring candidates by distance, cost, and available stock. The Distributed Inventory Counter Store holds the authoritative available/reserved/on-hand counts per warehouse per SKU and is the only component allowed to mutate a count, using an atomic operation so two concurrent requests can never both succeed against the same last unit. The Reservation Ledger is the durable system of record for every hold, tracking its state through held, committed, released, and expired, and is what a reconciliation job or a customer support agent can query after the fact. The TTL Expiry Sweeper actively finds holds whose TTL has lapsed and releases their stock back to the counter store, rather than waiting passively for someone to notice. The Sold-Out Signal Bus broadcasts availability deltas to the catalog and search layer so a listing page reflects “sold out” or “back in stock” within seconds of the counter store’s state actually changing.

A reservation’s life looks like this: a shopper starts checkout, the gateway assigns an idempotency key and passes the request to the routing engine, which picks one or more warehouses that together can cover the requested quantity. The gateway issues an atomic reserve against the counter store for each chosen warehouse, and only if every warehouse’s reserve succeeds does the gateway write a held row to the ledger with an expiry timestamp; if any warehouse’s reserve fails partway through a split allocation, the gateway rolls back the warehouses that already succeeded. From there, one of two things happens: the payment webhook confirms success and the gateway commits the reservation, permanently decrementing on-hand stock and marking the ledger row committed, or the TTL lapses without a commit and the sweeper releases the held quantity back to each warehouse’s available count and marks the ledger row expired. Every state change publishes an availability delta onto the signal bus, and once a SKU’s summed available count across all fifty warehouses hits zero, the bus fires a sold-out signal to the catalog.

Component Deep Dives

The Reservation Gateway and Idempotency Layer

This component’s job is to make sure the exact same client request, submitted twice because of a slow network or a retried mobile app, never creates two holds or triggers two releases.

The obvious mistake is trusting the client to only ever send a request once. Mobile networks drop connections mid-request constantly, and a client that times out waiting for a response has no way to know whether the server actually processed the request before the connection died, so a correct client has to retry, and a correct server has to treat that retry as harmless. We require every reserve, commit, and release call to carry a client-generated idempotency_key, and the gateway checks that key against a short-lived store before doing any work.

# Idempotency check performed before any reservation work begins
# Demonstrates: deduplicating a retried request using a Redis SETNX
# with the original response cached so a retry replays it verbatim
import json
import redis

IDEMPOTENCY_TTL_SECONDS = 86400  # keep replay protection for 24 hours

def with_idempotency(redis_client: redis.Redis, idempotency_key: str, handler):
    cache_key = f"idem:{idempotency_key}"
    lock_key = f"idem-lock:{idempotency_key}"

    cached = redis_client.get(cache_key)
    if cached is not None:
        return json.loads(cached)

    acquired = redis_client.set(lock_key, "1", nx=True, ex=30)
    if not acquired:
        # Another instance of this exact request is already in flight.
        raise DuplicateInFlightError(idempotency_key)

    try:
        result = handler()
        redis_client.set(cache_key, json.dumps(result), ex=IDEMPOTENCY_TTL_SECONDS)
        return result
    finally:
        redis_client.delete(lock_key)

Think of this like a coat check ticket at a busy restaurant: showing the same stub twice gets you the same coat handed back, never a second coat pulled off the rack. What breaks without it: a shopper on a spotty connection taps “reserve” once, the request goes through, the response is lost, the client retries, and now two holds exist for one checkout session, quietly eating into a SKU’s available count for stock that was never actually double-requested by a human.