Imagine a librarian responsible for 500 million books. Every second, some books go out of stock, prices change, and new bestseller lists arrive. A customer walks up and asks for “wireless headphones.” The librarian must, in 200 milliseconds, recall which books match, weigh which are in stock at the right price, factor in that this particular customer always buys Sony over Bose, and hand over a ranked list of 20 results - while simultaneously serving 100,000 other customers asking different questions. That is Amazon product search.

The challenge is not retrieval alone. Any search engine can find products matching a query. The hard part is that three forces are permanently in tension. Freshness demands that a product showing “In Stock” in search results is actually purchasable right now - Amazon loses revenue every time a customer clicks a result only to find it unavailable. But reindexing 500 million products fast enough to reflect every price and inventory change is expensive. Personalization demands that the same query “protein powder” surfaces whey isolate for a bodybuilder and plant-based options for a vegan - but per-user ranking at 100K QPS is a latency nightmare. And recall quality demands that a relevant product is never buried so deep it becomes invisible, even if it lacks the keyword density of competitors who optimize their listings.

The naive solution - build one big Elasticsearch cluster, query it at search time, and sort by a formula - fails at three different seams. Elasticsearch cannot ingest 500 million document updates fast enough to keep inventory fresh. A single ranking formula cannot personalize for 300 million users. And a static index cannot blend real-time signals like sales velocity without rebuilding the index on every change. We need a multi-stage architecture that separates recall from ranking, static signals from real-time signals, and global relevance from personal preference.

Requirements and Constraints

Functional Requirements

• Query-time ranking of products matching a search term, ordered by a relevance and conversion score
• Real-time signal injection: price, inventory availability, and sales velocity must reflect state within 30 seconds of change
• Per-user personalization: results must be re-ranked based on the individual user’s purchase and browse history
• Full catalog indexing: 500 million products must be discoverable; new products must appear in results within 5 minutes of creation

Non-Functional Requirements

• Latency: P99 under 200ms end-to-end including network; P50 under 80ms
• Throughput: 100,000 search requests per second at peak
• Catalog size: 500 million active product listings
• Signal freshness: Price and inventory signals must reflect real state within 30 seconds; sales velocity within 5 minutes
• Availability: 99.99% - search is on the revenue-critical path; every second of downtime costs thousands of dollars

Constraints and Scope

• Re-ranking is applied only to the top 1,000 candidates from the recall phase - not the entire matching set
• Spell correction, query understanding, and synonym expansion are upstream systems; we receive a clean tokenized query
• The advertising auction (sponsored products) runs in a separate system; we only rank organic results here
• User auth token is present on every search request; anonymous users receive global ranking with no personalization

High-Level Architecture

The system has five layers. The Search API accepts queries and orchestrates the pipeline. The Recall Engine retrieves up to 1,000 candidate products using an inverted index. The Feature Store provides real-time signals for each candidate. The Ranking Service scores and orders candidates using a learning-to-rank model. The Personalization Layer applies per-user adjustments to the final ranking before returning results.

The read path is strictly sequential within layers but parallel across sub-components. The Search API fans out the tokenized query to all relevant index shards simultaneously (scatter), collects up to 1,000 candidates (gather), then passes the candidate set to the Ranking Service. The Ranking Service calls the Feature Store in parallel for all 1,000 candidates in a single batch request, constructs feature vectors, runs the L2R model, then passes the scored list to the Personalization Layer.

The Catalog Indexing Pipeline

Every product in Amazon’s catalog starts as a row in a product database. To become searchable, it must be transformed into a set of index tokens and written into an inverted index shard. This is the catalog indexing pipeline, and it has two modes: a full rebuild that runs nightly, and an incremental delta path that keeps the index fresh in near-real-time.

The full rebuild pipeline reads the entire product catalog from the source database, extracts features (title tokens, category, brand, bullet points), computes static signals (historical sales rank, review count), builds inverted index segments, and deploys them to the index shard cluster. This runs on a Spark cluster over 4-6 hours, producing ~500GB of raw inverted index data.

The incremental path handles changes between full rebuilds. A change data capture (CDC) feed from the product database emits an event for every product that changes - price update, new description, inventory change, new review. The Feature Extractor processes these events and writes delta index segments. Delta segments are merged into the live index on a rolling basis.

# Index document builder - converts raw product record to index entry
from dataclasses import dataclass, field
from typing import List, Dict
import re

@dataclass
class ProductIndexDoc:
    product_id: str
    tokens: List[str]
    title: str
    brand: str
    category_path: List[str]
    static_score: float       # pre-computed at index time, rarely changes
    review_count: int
    avg_rating: float

def build_index_doc(product: dict) -> ProductIndexDoc:
    """
    Convert a raw product record into an index document.
    Static signals are computed here and stored in the index.
    Real-time signals (price, inventory) are NOT stored here -
    they are injected at query time from the Feature Store.
    """
    title_tokens = tokenize(product["title"])
    bullet_tokens = []
    for bullet in product.get("bullet_points", []):
        bullet_tokens.extend(tokenize(bullet))

    # De-duplicate tokens, keep title tokens weighted higher
    all_tokens = list(set(title_tokens + bullet_tokens))

    # Static quality score: function of review count and rating
    review_count = product.get("review_count", 0)
    avg_rating = product.get("avg_rating", 0.0)
    static_score = compute_static_score(review_count, avg_rating)

    return ProductIndexDoc(
        product_id=product["asin"],
        tokens=all_tokens,
        title=product["title"],
        brand=product.get("brand", ""),
        category_path=product.get("category_path", []),
        static_score=static_score,
        review_count=review_count,
        avg_rating=avg_rating,
    )

def tokenize(text: str) -> List[str]:
    """Lowercase, strip punctuation, split on whitespace."""
    text = text.lower()
    text = re.sub(r"[^a-z0-9\s]", " ", text)
    return [t for t in text.split() if len(t) > 1]

def compute_static_score(review_count: int, avg_rating: float) -> float:
    """Wilson lower bound approximation for review quality signal."""
    if review_count == 0:
        return 0.0
    import math
    z = 1.96  # 95% confidence
    n = review_count
    p = avg_rating / 5.0
    lower = (p + z*z/(2*n) - z * math.sqrt((p*(1-p) + z*z/(4*n))/n)) / (1 + z*z/n)
    return lower

Index freshness is the hardest operational challenge here. The delta path must process product changes fast enough that inventory and pricing in the index never diverge more than 30 seconds from reality. In practice, price and inventory signals are so volatile that they are never stored in the inverted index at all - they live exclusively in the Feature Store and are injected at query time. Only the structural product data (title, description, category) lives in the inverted index, which changes slowly enough for the delta path to keep up.