A summer camp with six basketball courts and one counselor running pickup games doesn’t seat kids by arrival order alone. The counselor is juggling three things at once: skill (nobody wants a game where one team is all twelve-year-old varsity players and the other is complete beginners), which court is actually close by (a kid isn’t walking fifteen minutes to the far court when three closer ones are open), and how long a kid has already been standing on the sideline. Get the skill match wrong and the game is over in two minutes, one team steamrolling the other. Get the proximity wrong and half the kids give up walking before they reach the court. Ignore the sideline clock and the same unlucky, oddly-skilled kid waits all afternoon while everyone else gets picked.

A game matchmaking service runs the exact same juggling act, except the courts are data centers, the skill ranking is an ELO rating, and the sideline clock is a queue wait time SLO measured in seconds instead of afternoons. At 800,000 concurrent players queueing for 5v5 matches across dozens of regions, the system has to pull compatible tickets out of a pool of tens of thousands of waiting players, evaluate every candidate along skill, latency, and wait time simultaneously, and hand off a fully formed, fair, low-lag match to a dedicated game server, typically within 45 seconds at the 95th percentile.

The naive version of this problem looks deceptively easy: sort the queue by arrival time and match the first ten people who show up. That’s a FIFO queue, and it produces exactly the kind of stomp the counselor is trying to avoid; a brand-new player at 900 ELO and a five-year veteran at 2400 end up on opposite teams by pure coincidence of timing. Sort by skill instead, with a narrow fixed window (say, plus or minus 50 ELO), and the fairness problem is fixed, but a new one appears: in any bracket outside the dense middle of the skill distribution, in an off-peak region, at 3am local time, there simply may not be nine other players within 50 ELO points online right now. That ticket waits forever, and forever is not an SLO. Add latency into the equation and it gets worse still: two players can be a perfect skill match and sit on opposite sides of an ocean, and forcing them into the same match produces an unplayable 180ms-round-trip game even though the ELO numbers looked great on paper.

The forces in real tension here are fairness (the closest possible skill match), playability (the lowest possible latency to the server hosting the match), and patience (a bounded queue wait, no matter how thin a particular skill bracket, region, or hour happens to be), and none of them can be solved in isolation, because tightening any one of them starves the others. We need to solve for organizing hundreds of thousands of in-flight tickets into a structure that’s searchable in milliseconds, a single quality score that lets wildly different tickets (skill delta, latency, party size, minutes waited) be compared on equal footing, and a principled, monotonic way to relax the fairness bar over time so the wait-time SLO never gets breached even in the thinnest bracket, all simultaneously.

Requirements and Constraints

Functional Requirements

• Solo players and pre-made parties of 2 to 5 players can join a matchmaking queue for a specific game mode and region
• The system computes and updates every player’s ELO skill rating immediately after a match completes
• The system matches tickets into two balanced 5-player teams by jointly considering skill rating, latency to candidate game servers, and how long each ticket has already been queued
• A ticket’s acceptable skill and latency window widens automatically the longer it waits, so no ticket waits indefinitely for a match that will never appear
• Parties are matched as an atomic, indivisible unit, never split across opposing teams, and are matched preferentially against other parties before being paired against solo-only teams
• The system selects the game server region and cluster with the lowest worst-case latency across all ten matched players and allocates a dedicated server session
• Clients receive a live queue status stream (elapsed wait, estimated time remaining) while their ticket is searching
• A player or party can cancel their ticket at any point before a match is confirmed, cleanly releasing it from the pool

Non-Functional Requirements

• Peak scale: 800,000 concurrent players, roughly 750 tickets/sec sustained ticket ingress at peak, roughly 75 matches formed/sec at peak
• Queue wait SLO: p95 under 45 seconds and p99 under 90 seconds, measured per (region, game_mode, skill_bracket)
• Match fairness: average inter-team skill delta under 60 ELO points for at least 95% of matches formed inside the SLO window
• Ticket durability: a matchmaker worker crash must not silently drop an in-flight ticket; the ticket resumes searching, with its accumulated wait time preserved, within 2 seconds
• Rating update latency: ELO changes are computed and durably persisted within 2 seconds of match completion
• Availability: matchmaking pool availability of 99.9%; a regional outage reroutes affected tickets to the next-nearest healthy region rather than blocking matchmaking entirely
• Latency freshness: no match is formed using a per-player latency sample older than 30 seconds

Constraints and Assumptions

• Out of scope: the actual game server simulation and netcode, anti-cheat, voice chat, and the in-game economy. This design covers matchmaking and the trigger to allocate a server, not the game itself
• We assume clients report round-trip latency probes honestly; detecting probe spoofing or queue-dodging abuse is flagged as a real concern but not designed in depth here
• We assume a pre-existing dedicated game server fleet and orchestrator (an Agones-style system) that this design allocates sessions from, rather than building the orchestrator itself
• This design targets a single game mode family (ranked 5v5) at a time; supporting many simultaneous modes is discussed briefly in Scaling and Performance but not designed exhaustively
• We assume every match has a fixed size of 10 players (5v5); variable team sizes are a straightforward extension, not the primary design target

High-Level Architecture

The system has six major components: the Matchmaking Gateway (accepts tickets from solo players and parties, streams queue status back to clients), the Latency Probe Service (measures and maintains a per-player RTT estimate to candidate game server clusters), the Matchmaking Pool (an in-memory, region-and-skill-sharded structure holding every in-flight ticket), the Match Scoring and Assembly Engine (the background workers that search the pool, expand brackets over time, score candidates, and draft balanced teams), the Game Server Allocator (picks the best region for a formed match and requests a session from the dedicated server fleet), and the Post-Match Rating Service (ingests match results and updates ELO ratings).

A ticket’s journey starts at the Gateway, which validates the request (solo or party), computes an effective ELO rating for it, and writes it into the Matchmaking Pool shard for its region and skill bucket. In parallel, the Latency Probe Service keeps a rolling RTT estimate for every player against a handful of nearby server clusters, refreshed roughly every 10 seconds, so the Assembly Engine always has a recent latency picture to work with. The Assembly Engine continuously scans pool shards, expanding each ticket’s acceptable skill and latency window the longer it has waited, scoring every candidate set it finds against a single quality function, and drafting compatible tickets (parties kept intact) into two 5-player teams once a candidate set clears the quality bar. Once a team pair is formed, the Allocator runs a short ready-check, picks the server cluster that minimizes the worst latency any single player would see, and hands the match off to a warm dedicated server. When the match ends, the Post-Match Rating Service applies the ELO update for all ten players and closes out the audit trail used to measure the queue wait SLO.

The Matchmaking Pool and Ticket Queue

This component’s job is to hold every in-flight ticket in a structure a background worker can filter by region, skill, and elapsed wait time in milliseconds, without scanning the whole pool on every search tick.

A smart engineer’s first instinct is a single global queue, either FIFO or sorted once by ELO at insert time. Both break down fast. FIFO ignores skill entirely, which we already covered. A single sorted structure across every region and every skill level creates contention on one hot data structure and forces every search to skip over players an ocean away or leagues out of range just to find the handful that matter. What actually works is the same idea a library uses for a card catalog: don’t search the whole library, walk straight to the drawer for the right subject and call number range.

We shard the pool by (region, game_mode, coarse_elo_bucket), with each bucket spanning 200 ELO points, backed by a Redis sorted set per shard. The sorted set’s score is the ticket’s elo_rating, so a bracket search is a single ZRANGEBYSCORE call, O(log N + M) where M is the number of tickets actually in range, never a scan of the full pool.

// Matchmaking Gateway -> Pool RPC contract
// Demonstrates: ticket submission shape for solo players and parties
syntax = "proto3";

package matchmaking.v1;

message TicketRequest {
  string requester_id = 1;        // player_id or party_id
  repeated string player_ids = 2; // 1 for solo, 2-5 for a party
  string region = 3;              // preferred home region, e.g. "na-east"
  string game_mode = 4;           // e.g. "ranked_5v5"
  int32 party_elo = 5;            // pre-computed effective party rating
}

message TicketResponse {
  string ticket_id = 1;
  int64 queued_at_ms = 2;
  int32 initial_search_radius = 3;
}

service MatchmakingGateway {
  rpc SubmitTicket(TicketRequest) returns (TicketResponse);
  rpc CancelTicket(CancelRequest) returns (CancelResponse);
}

message CancelRequest {
  string ticket_id = 1;
}

message CancelResponse {
  bool cancelled = 1;
}

# Matchmaking Pool: region and bracket sharded ticket queue backed by Redis
# Demonstrates: ticket insert, bracket-window search, and multi-bucket fan-out
import redis

BRACKET_WIDTH = 200  # ELO points per coarse bucket, used only for shard routing

class TicketPoolShard:
    # One Redis sorted set per (region, mode, elo bucket). Score is elo_rating,
    # so ZRANGEBYSCORE gives every ticket within a skill window in O(log N + M).
    def __init__(self, client: redis.Redis, region: str, mode: str, elo_bucket: int):
        self.client = client
        self.key = f"pool:{region}:{mode}:{elo_bucket}"
        self.meta_prefix = f"ticket_meta:{region}:{mode}:{elo_bucket}"

    def insert(self, ticket_id: str, elo: int, queued_at_ms: int, party_size: int) -> None:
        self.client.zadd(self.key, {ticket_id: elo})
        self.client.hset(f"{self.meta_prefix}:{ticket_id}", mapping={
            "queued_at_ms": queued_at_ms,
            "party_size": party_size,
            "elo": elo,
        })

    def search_window(self, elo: int, radius: int) -> list[str]:
        # Every ticket whose rating falls inside [elo - radius, elo + radius]
        return self.client.zrangebyscore(self.key, elo - radius, elo + radius)

    def remove(self, ticket_id: str) -> None:
        self.client.zrem(self.key, ticket_id)
        self.client.delete(f"{self.meta_prefix}:{ticket_id}")

def adjacent_buckets(elo: int, radius: int, bracket_width: int = BRACKET_WIDTH) -> list[int]:
    # A wide search radius can span more than one coarse bucket; return every
    # bucket index the window touches so the caller can fan out across shards
    low_bucket = (elo - radius) // bracket_width
    high_bucket = (elo + radius) // bracket_width
    return list(range(low_bucket, high_bucket + 1))

What would break if we simplified this to one global sorted set is contention, not correctness: every insert, every search, and every removal would serialize against the same key, and a popular mid-ELO bracket in a busy region would starve everyone else’s search latency. Sharding by region and coarse bucket means a query only ever touches the handful of shards its current radius actually overlaps, and unrelated brackets never see each other’s load at all.

The Latency Probe Service and Region Routing

This component’s job is to measure real round-trip latency between each client and the pool of candidate game server clusters, so matches route to the server that will actually feel good to play on, not just the one that’s geographically closest.

The naive assumption is that geo-IP based region assignment is close enough. It isn’t: VPNs, undersea cable routing, and ISP peering agreements routinely make real RTT diverge sharply from geographic distance, sometimes by more than 100ms in either direction. What we actually want is the same trick cell networks use to triangulate a phone’s position from tower signal strength rather than trusting GPS, which can be wrong or unavailable indoors: measure the real signal directly instead of inferring it from a proxy.

Each client actively pings a small set of nearby server clusters (typically 4 to 6) over a lightweight UDP echo every 10 seconds. Results are smoothed with an exponential moving average so a single noisy sample doesn’t whipsaw the estimate, and the smoothed value is what actually gets attached to the player’s ticket.

// Latency Probe Service: exponential moving average smoothing of client RTT samples
// Demonstrates: rolling latency estimate per player per candidate server cluster
package latency

import "time"

const emaAlpha = 0.3 // weight given to each new sample; higher = more reactive, less smooth

type Sample struct {
	ClusterID string
	RTTMillis float64
	SampledAt time.Time
}

type ClusterEstimate struct {
	SmoothedRTT float64
	LastSample  time.Time
	Confidence  string // "measured" or "geoip_fallback"
}

// UpdateEstimate folds a fresh RTT sample into the existing smoothed estimate. The
// first real sample for a cluster seeds the average directly instead of blending
// it against a geoIP guess, so one good ping immediately beats the fallback.
func UpdateEstimate(existing *ClusterEstimate, sample Sample) *ClusterEstimate {
	if existing == nil || existing.Confidence == "geoip_fallback" {
		return &ClusterEstimate{
			SmoothedRTT: sample.RTTMillis,
			LastSample:  sample.SampledAt,
			Confidence:  "measured",
		}
	}
	smoothed := emaAlpha*sample.RTTMillis + (1-emaAlpha)*existing.SmoothedRTT
	return &ClusterEstimate{
		SmoothedRTT: smoothed,
		LastSample:  sample.SampledAt,
		Confidence:  "measured",
	}
}

// IsFresh enforces the 30-second latency freshness requirement before a cluster's
// estimate is allowed to influence match server selection.
func IsFresh(e *ClusterEstimate, now time.Time, maxAge time.Duration) bool {
	return e != nil && now.Sub(e.LastSample) <= maxAge
}

A brand-new player has no samples yet. Rather than blocking their first ticket on a probe round trip, we seed their estimate from a geoIP lookup, flagged confidence: geoip_fallback, and let the first real UDP probe overwrite it outright the moment it lands, instead of blending the two. The Assembly Engine treats geoip_fallback estimates as weaker evidence in the quality score, described later, so a cold-start player doesn’t get an artificially confident match while their real latency is still unknown.

Party Matchmaking

This component’s job is to treat a group of 2 to 5 players who queued together as one indivisible unit, so a match never splits friends across opposing teams, while still producing a fair overall match.

The naive assumption is that a party’s effective rating is just the average of its members’ ELO. That’s wrong in two ways. First, it ignores that pre-made coordination is a real, measurable advantage over five strangers who’ve never voiced together, so a party should be matched slightly stiffer than a raw average suggests. Second, a plain average lets a wide-variance party (a 2600-rated player queued with a 1200-rated friend) hide behind a comfortable-looking midpoint average, when in practice that party plays very differently than five solo players clustered tightly around the same number.

# Party Matchmaking: effective party rating and party-aware team assembly
# Demonstrates: synergy-adjusted party rating, snake-draft team balancing that
# keeps every party intact as a single indivisible block
from dataclasses import dataclass

@dataclass
class Ticket:
    ticket_id: str
    player_ids: list[str]
    elo_rating: int          # effective rating for this ticket, party or solo
    queued_at_ms: int

def party_effective_rating(member_elos: list[int]) -> int:
    # Weighted average plus a synergy adjustment: pre-made coordination is worth
    # a real edge, so a party is matched slightly stiffer than a raw average. The
    # bump is capped and offset by a variance penalty so a wide-spread party can't
    # hide a strong player behind a deliberately low-rated one to sandbag a match.
    avg = sum(member_elos) / len(member_elos)
    variance_penalty = (max(member_elos) - min(member_elos)) * 0.05
    synergy_bump = min(40, 8 * (len(member_elos) - 1))
    return round(avg + synergy_bump - variance_penalty)

def assemble_teams(tickets: list[Ticket], team_size: int = 5) -> tuple[list[Ticket], list[Ticket]] | None:
    # Treats every ticket (solo player or intact party) as an indivisible block and
    # snake-drafts them onto two teams, highest rating first, minimizing the skill
    # gap between team totals while never splitting a party.
    pool = sorted(tickets, key=lambda t: t.elo_rating, reverse=True)
    team_a: list[Ticket] = []
    team_b: list[Ticket] = []
    size_a = size_b = 0

    for ticket in pool:
        n = len(ticket.player_ids)
        fits_a = size_a + n <= team_size
        fits_b = size_b + n <= team_size
        if not fits_a and not fits_b:
            continue  # doesn't fit either team in this candidate set, try the next window
        sum_a = sum(t.elo_rating for t in team_a)
        sum_b = sum(t.elo_rating for t in team_b)
        if fits_a and (not fits_b or sum_a <= sum_b):
            team_a.append(ticket)
            size_a += n
        else:
            team_b.append(ticket)
            size_b += n

    if size_a == team_size and size_b == team_size:
        return team_a, team_b
    return None  # candidate set doesn't cleanly fill both teams, caller widens the search

Think of team assembly as draft day: two captains alternate picking the best remaining player, except some players insist on being drafted together as a package deal. The snake-draft loop above always hands the next ticket to whichever team currently has the lower running total, which is a simple, fast greedy heuristic that keeps the two team sums close without ever needing to solve the (NP-hard, in general) exact partition problem, because a 10-player candidate window is small enough that greedy plus the quality-score check downstream catches anything the heuristic leaves imbalanced.

The Assembly Engine also prefers matching a party against another party of similar total size and rating before it falls back to pairing a party against a team of five solo queuers, specifically to reduce “party stacking,” the perception (often justified) that a coordinated five-stack has an unfair edge over five strangers. That preference is soft, not absolute: once a ticket’s wait time pushes its constraints wide enough, party-vs-solo matches become acceptable rather than leaving the party waiting indefinitely for a same-size opponent that may not exist in a thin bracket.

Match Assembly and Game Server Allocation

This component’s job is to take a candidate set of tickets that already cleared the quality bar, confirm every player is still actually there, and hand the match off to a real dedicated game server before anyone’s patience or connection runs out.

The naive assumption is that once teams are picked, you can immediately allocate a server and start the match. Two things go wrong if you do. First, someone in that candidate set may have already disconnected, alt-tabbed away, or cancelled since they were last seen, and spinning up a server for nine people and a ghost wastes a warm server slot and forces an awkward mid-load abort. Second, naively allocating from whichever region the majority of players prefer can produce a great average latency with one unlucky player stuck at 220ms, which is an unplayable match for that one person even though the aggregate numbers look fine.

// Match Assembly: ready-check reservation and worst-case-latency server allocation
// Demonstrates: confirming every player before spinning a server, and picking the
// region that minimizes the *worst* latency across the match, not the average
package assembly

import (
	"errors"
	"time"
)

type ConfirmedPlayer struct {
	PlayerID string
	Latency  map[string]float64 // clusterID -> smoothed RTT ms
}

var ErrNoServerAvailable = errors.New("no warm server available in any candidate region")

// BestCluster picks the candidate cluster that minimizes the worst latency any
// single player would experience, not the average - a great average latency with
// one player at 220ms still produces an unplayable match for that one player.
func BestCluster(players []ConfirmedPlayer, candidateClusters []string) (string, error) {
	best := ""
	bestWorst := 1 << 30
	for _, cluster := range candidateClusters {
		worst := 0
		for _, p := range players {
			rtt, ok := p.Latency[cluster]
			if !ok {
				worst = 1 << 30 // missing data disqualifies this cluster for this player
				break
			}
			if int(rtt) > worst {
				worst = int(rtt)
			}
		}
		if worst < bestWorst {
			bestWorst = worst
			best = cluster
		}
	}
	if best == "" {
		return "", ErrNoServerAvailable
	}
	return best, nil
}

// ReadyStore is a small interface over the Redis hash that tracks ready-acks
// for an in-flight match, keyed by matchID.
type ReadyStore interface {
	AckedPlayers(matchID string) ([]string, error)
}

// AwaitReadyCheck polls the ready store until every expected player has acked or
// readyTimeout elapses, whichever comes first, then returns who confirmed and
// who needs to be requeued with their accumulated wait time preserved.
func AwaitReadyCheck(store ReadyStore, matchID string, expected []string, readyTimeout time.Duration) (confirmed, dropped []string) {
	deadline := time.Now().Add(readyTimeout)
	ticker := time.NewTicker(200 * time.Millisecond)
	defer ticker.Stop()

	acked := map[string]bool{}
	for time.Now().Before(deadline) {
		ids, err := store.AckedPlayers(matchID)
		if err == nil {
			for _, id := range ids {
				acked[id] = true
			}
		}
		if len(acked) == len(expected) {
			break
		}
		<-ticker.C
	}

	for _, id := range expected {
		if acked[id] {
			confirmed = append(confirmed, id)
		} else {
			dropped = append(dropped, id)
		}
	}
	return confirmed, dropped
}

If a player fails to ready-up, the match doesn’t just die quietly. The remaining nine players get requeued as a smaller candidate set, and critically, their queued_at_ms is untouched, so their bracket-expansion progress carries forward instead of resetting to zero. The player who failed to ready gets a short matchmaking cooldown, which discourages queue-dodging (accepting a ticket, then bailing at the last second to re-roll a better match) without punishing an honest disconnect too harshly.

Data Model

The schema separates four concerns: durable player state, the historical record of completed matches, the append-only ELO change log, and the rolling latency estimates. Live in-flight tickets deliberately do not live here; they live in the Redis-backed pool described above, because they’re ephemeral, high-churn, and only need durability on the order of seconds, not the durability guarantees a relational store provides.

-- Players: durable profile and current skill rating
CREATE TABLE players (
    player_id       BIGINT       PRIMARY KEY,
    display_name    TEXT         NOT NULL,
    region          TEXT         NOT NULL,     -- home region, seeds initial pool routing
    elo_rating      INTEGER      NOT NULL DEFAULT 1200,
    games_played    INTEGER      NOT NULL DEFAULT 0,
    is_provisional  BOOLEAN      NOT NULL DEFAULT TRUE,  -- true until games_played >= 30
    created_at      TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    updated_at      TIMESTAMPTZ  NOT NULL DEFAULT NOW()
);

CREATE INDEX idx_players_region_elo ON players (region, elo_rating);

-- Matches: one row per formed match
CREATE TABLE matches (
    match_id         BIGINT       PRIMARY KEY,
    region           TEXT         NOT NULL,
    game_mode        TEXT         NOT NULL,
    server_cluster   TEXT         NOT NULL,
    quality_score    NUMERIC(5,4) NOT NULL,   -- 0.0000 - 1.0000
    avg_skill_delta  NUMERIC(6,2) NOT NULL,   -- inter-team ELO delta
    formed_at        TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    completed_at     TIMESTAMPTZ
);

CREATE INDEX idx_matches_region_formed ON matches (region, formed_at DESC);

-- Match participants: who was in a match, which team, and their pre-match rating
CREATE TABLE match_participants (
    match_id        BIGINT    NOT NULL REFERENCES matches(match_id),
    player_id       BIGINT    NOT NULL REFERENCES players(player_id),
    team            SMALLINT  NOT NULL CHECK (team IN (0, 1)),
    party_id        BIGINT,                     -- NULL for solo queuers
    pre_match_elo   INTEGER   NOT NULL,
    wait_time_ms    INTEGER   NOT NULL,
    PRIMARY KEY (match_id, player_id)
);

CREATE INDEX idx_participants_player ON match_participants (player_id);

-- ELO history: append-only log of every rating change, partitioned by month
CREATE TABLE elo_history (
    id          BIGINT       GENERATED ALWAYS AS IDENTITY,
    player_id   BIGINT       NOT NULL REFERENCES players(player_id),
    match_id    BIGINT       NOT NULL REFERENCES matches(match_id),
    elo_before  INTEGER      NOT NULL,
    elo_after   INTEGER      NOT NULL,
    k_factor    SMALLINT     NOT NULL,
    created_at  TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    PRIMARY KEY (id, created_at)
) PARTITION BY RANGE (created_at);

CREATE INDEX idx_elo_history_player ON elo_history (player_id, created_at DESC);

-- Latency samples: rolling per-player, per-cluster smoothed RTT estimate
CREATE TABLE latency_samples (
    player_id        BIGINT       NOT NULL REFERENCES players(player_id),
    cluster_id       TEXT         NOT NULL,
    smoothed_rtt_ms  NUMERIC(6,2) NOT NULL,
    confidence       TEXT         NOT NULL DEFAULT 'measured',  -- measured | geoip_fallback
    sampled_at       TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    PRIMARY KEY (player_id, cluster_id)
);

CREATE INDEX idx_latency_stale ON latency_samples (sampled_at);

-- Ticket audit: append-only lifecycle log used to compute wait-time percentiles,
-- partitioned by day since it is high volume and only needed for a rolling window
CREATE TABLE ticket_audit (
    ticket_id    TEXT         NOT NULL,
    event_type   TEXT         NOT NULL,  -- queued | radius_expanded | matched | cancelled | timed_out
    region       TEXT         NOT NULL,
    game_mode    TEXT         NOT NULL,
    elo_bucket   INTEGER      NOT NULL,
    wait_time_ms INTEGER,
    event_at     TIMESTAMPTZ  NOT NULL DEFAULT NOW()
) PARTITION BY RANGE (event_at);

CREATE INDEX idx_ticket_audit_slo ON ticket_audit (region, game_mode, elo_bucket, event_at);

elo_history and ticket_audit are both partitioned by time because they’re append-only, high volume, and only need to be hot for a rolling window (90 days for rating history, 30 days for SLO measurement), after which old partitions are archived to cold storage and dropped rather than bloating the primary index. match_participants is indexed by player_id because “show me a player’s match history” is the dominant read pattern outside of matchmaking itself, and it stores pre_match_elo and wait_time_ms per row specifically so fairness and SLO metrics can be recomputed later without needing to replay the live pool state, which by then no longer exists.

A ticket moves through a small, well-defined state machine over its life: queued (written into a pool shard, wait clock starts), searching (radius widens on a fixed tick until a candidate window appears), quality verified (the candidate set clears the dynamic quality floor), teams assembled (party-aware draft into two sides), ready check (every player confirms), and finally server allocated and completed, at which point the Post-Match Rating Service closes the loop with an ELO update. The diagram traces that lifecycle and the one exception path, a failed ready check, which returns players to Searching without resetting their wait clock.

Key Algorithms and Protocols

ELO Rating Updates

Each player’s skill rating updates immediately after every match using the classic ELO formula: a player’s expected score against an opponent is a logistic function of the rating gap, and the actual outcome (win or loss) pulls the rating toward or away from that expectation, scaled by a K-factor.

# ELO Rating Update: classic ELO with a dynamic K-factor by experience tier
# Demonstrates: expected score calculation and per-player rating update after a match
def expected_score(rating_a: float, rating_b: float) -> float:
    # Probability rating A is expected to win against B, from the logistic curve
    # ELO is built on. Values are naturally bounded to (0, 1) by the exponent.
    return 1.0 / (1.0 + 10 ** ((rating_b - rating_a) / 400))

def k_factor(games_played: int, rating: int) -> int:
    if games_played < 30:
        return 40   # provisional: let new ratings move fast toward their true skill
    if rating >= 2400:
        return 10   # top of the ladder: dampen swings so ranks stay stable
    return 20       # established players in the broad middle of the distribution

def update_rating(rating: int, games_played: int, opponent_avg_rating: int, won: bool) -> int:
    k = k_factor(games_played, rating)
    expected = expected_score(rating, opponent_avg_rating)
    actual = 1.0 if won else 0.0
    delta = k * (actual - expected)
    return round(rating + delta)

For a 5v5 team match, each player’s individual update_rating call uses the opposing team’s average rating as the opponent rating, so a player’s rating still moves based on how their whole team performed relative to the other team as a whole, not on any single 1-on-1 comparison that doesn’t exist in a team game.

# Team ELO update: apply the per-player formula using team averages, with an
# optional small per-player performance modifier
def update_team_ratings(team: list[dict], opponent_team: list[dict], team_won: bool,
                         perf_modifiers: dict[str, float] | None = None) -> dict[str, int]:
    opponent_avg = sum(p["elo_rating"] for p in opponent_team) / len(opponent_team)
    updates = {}
    for player in team:
        base_new_rating = update_rating(
            rating=player["elo_rating"],
            games_played=player["games_played"],
            opponent_avg_rating=round(opponent_avg),
            won=team_won,
        )
        modifier = 1.0
        if perf_modifiers:
            modifier = perf_modifiers.get(player["player_id"], 1.0)
        delta = round((base_new_rating - player["elo_rating"]) * modifier)
        updates[player["player_id"]] = player["elo_rating"] + delta
    return updates

Edge cases worth naming: a brand-new player’s is_provisional flag (backed by games_played < 30) keeps their K-factor high so their rating converges quickly toward their true skill instead of anchoring near the default 1200 for dozens of games; a top-of-ladder player’s K-factor is deliberately dampened so a single upset doesn’t swing a hard-earned rank; and expected_score is naturally bounded, so even an extreme rating gap (a 900 versus a 2800) never produces a nonsensical expected value outside (0, 1).

Skill Bracket Expansion Over Time

This is the mechanism that directly protects the queue wait SLO: as a ticket waits longer, both its acceptable skill window and its acceptable latency ceiling grow, monotonically, until the ticket is effectively willing to accept almost anyone.

# Skill and latency bracket expansion: search constraints widen monotonically
# the longer a ticket waits, bounding worst-case queue time without ever
# tightening back up mid-search
BASE_ELO_RADIUS = 50
ELO_GROWTH_PER_SEC = 8
MAX_ELO_RADIUS = 600

BASE_LATENCY_MS = 40
LATENCY_GROWTH_PER_SEC = 2.0
MAX_LATENCY_MS = 150

def elo_search_radius(wait_seconds: float) -> int:
    radius = BASE_ELO_RADIUS + ELO_GROWTH_PER_SEC * wait_seconds
    return min(MAX_ELO_RADIUS, round(radius))

def latency_ceiling_ms(wait_seconds: float) -> float:
    ceiling = BASE_LATENCY_MS + LATENCY_GROWTH_PER_SEC * wait_seconds
    return min(MAX_LATENCY_MS, round(ceiling, 1))

def is_open_match(wait_seconds: float) -> bool:
    # Once both constraints hit their ceiling, the ticket accepts effectively any
    # candidate that clears a minimum sanity bar - this is what guarantees the
    # wait-time SLO can never be breached by an unmatched ticket
    return (elo_search_radius(wait_seconds) >= MAX_ELO_RADIUS
            and latency_ceiling_ms(wait_seconds) >= MAX_LATENCY_MS)

With these constants, the ELO radius reaches its 600-point ceiling at t = (600 - 50) / 8 = 68.75 seconds, and the latency ceiling reaches its 150ms cap at t = (150 - 40) / 2 = 55 seconds. Both fully open well before the p99 target of 90 seconds, which leaves a deliberate buffer: a ticket that’s been open for the full 68.75 seconds and still hasn’t matched means the pool itself is critically thin (an outage, a tiny off-peak region), not that the expansion curve needs more room. At the p95 target of 45 seconds, the radius has grown to 50 + 8*45 = 410 ELO and the latency ceiling to 40 + 2*45 = 130ms, both comfortably mid-expansion, not fully open, so most matches inside the SLO still have meaningfully constrained quality.

Match Quality Scoring

Once the pool search returns a candidate window, the Assembly Engine needs a single number to decide whether a given candidate set is worth forming into a match right now, or whether to keep searching. That means combining three signals measured in completely different units, ELO points, milliseconds, and seconds, into one comparable score.

# Match Quality Scoring: combine skill delta, worst-case latency, and wait time
# into one normalized [0, 1] score so wildly different tickets can be compared
def normalize(value: float, worst: float, best: float = 0.0) -> float:
    # Maps value into [0, 1] where 1.0 is best (closest to `best`) and 0.0 is worst
    span = worst - best
    if span <= 0:
        return 1.0
    return max(0.0, min(1.0, 1.0 - (value - best) / span))

SKILL_WEIGHT = 0.5
LATENCY_WEIGHT = 0.3
WAIT_WEIGHT = 0.2

def match_quality(skill_delta: float, worst_latency_ms: float, avg_wait_seconds: float) -> float:
    skill_component = normalize(skill_delta, worst=200.0)          # 200+ ELO delta scores 0
    latency_component = normalize(worst_latency_ms, worst=150.0)   # 150ms+ scores 0
    wait_component = normalize(90.0 - avg_wait_seconds, worst=90.0) # longer wait -> higher score

    return (SKILL_WEIGHT * skill_component
            + LATENCY_WEIGHT * latency_component
            + WAIT_WEIGHT * wait_component)

MIN_ACCEPTABLE_QUALITY = 0.55

def is_acceptable(skill_delta: float, worst_latency_ms: float, avg_wait_seconds: float) -> bool:
    quality = match_quality(skill_delta, worst_latency_ms, avg_wait_seconds)
    # The bar itself relaxes slightly as wait grows, paired with bracket expansion,
    # so a match that's merely "good enough" late in the search still clears it
    dynamic_floor = max(0.35, MIN_ACCEPTABLE_QUALITY - 0.002 * avg_wait_seconds)
    return quality >= dynamic_floor

skill_component and latency_component reward tight matches, while wait_component deliberately does the opposite: it grows as avg_wait_seconds grows, so the same candidate set becomes more attractive to accept purely because the tickets in it have been waiting longer. That’s what stops the Assembly Engine from endlessly holding out for a theoretically better match while a real, acceptable one sits unformed in front of it.

Scaling and Performance

The pool scales along two independent axes: adding more (region, mode, bracket) shards as the player base or the number of supported modes grows, and adding more stateless matcher workers that lease shard ranges to search. Because a shard lease (acquired via a Redis SET NX style lock or an equivalent distributed lock) guarantees exactly one worker searches a given bucket range at a time, adding workers is a pure capacity increase with no coordination overhead beyond lease renewal.

Capacity Estimation:

Given:
  - 800,000 peak concurrent players (CCU) globally
  - 10-player matches (5v5), average match duration 16 minutes (960s)
  - ~90% of peak CCU (720,000 players) already inside an active match at any instant
  - Queue wait SLO: p95 under 45s, p99 under 90s

Concurrent active matches (Little's Law):
  720,000 players / 10 players per match = 72,000 concurrent matches

Match completion / ticket re-entry rate:
  72,000 matches / 960s average duration ~= 75 matches/sec completing at peak
  75 matches/sec * 10 players/match ~= 750 tickets/sec re-entering the pool (steady state)

Tickets in flight in the pool (Little's Law again, avg wait ~15s, below the p95 bound):
  750 tickets/sec * 15s average wait ~= 11,250 tickets in the pool at any instant

Pool memory footprint:
  11,250 tickets * ~300 bytes (ZSET entry + metadata hash) ~= 3.4 MB
  Sharded across ~50 active (region, mode, bracket) shards ~= ~68 KB average per shard,
  comfortably in-memory even before accounting for hot-shard sub-bucketing

Dedicated game server fleet:
  72,000 concurrent matches + 15% warm buffer for allocation latency ~= ~83,000 server instances

Latency probe bandwidth:
  ~200,000 players actively probing (queued + recently in menu) at any instant
  6 candidate clusters probed every 10s, ~128 bytes round-trip overhead per probe
  200,000 * 6 / 10s * 128 bytes ~= 15.4 MB/sec aggregate probe traffic, trivial for UDP echo

ELO history growth:
  ~40 matches/sec average (about half of peak) ~= 3.5M matches/day ~= 35M elo_history rows/day
  35M rows * ~80 bytes/row ~= 2.8 GB/day, 90-day retention ~= ~250 GB before archival

Read/write ratio isn’t the interesting hot spot here; every ticket insert is followed almost immediately by repeated reads (each expansion tick re-queries the same shard), so the real risk is shard skew, not read pressure in aggregate. A popular mid-ELO bracket in a busy region during peak hours absorbs far more load than an average shard, so BRACKET_WIDTH isn’t fixed globally; a hot bucket gets dynamically split into narrower sub-ranges with more lease workers assigned, the same load-aware rebalancing shown in the diagram above, while cold buckets stay wide and cheap. A region-level outage (the game server fleet in one region degrading) triggers a separate failover: new tickets destined for that region get rerouted to the next-nearest healthy region rather than piling up against a fleet that can’t allocate servers.

Failure Modes and Recovery

Failure	Detection	Impact	Recovery
Matchmaker worker crash mid-search (owns a shard lease)	Lease heartbeat missed by the lock service	Tickets in that shard stop making expansion progress	Lease reassigned to a healthy worker; search resumes using the stored queued_at_ms, wait time never resets
Redis pool shard node failure or network partition	Connection errors and failed health checks	Tickets in that shard become unsearchable	Replica promotion (Redis Sentinel/Cluster); client-side reconnect logic resubmits with the same idempotent ticket_id if the ticket isn’t found within a grace window, avoiding a duplicate ELO-affecting match
Game server allocation fails (no warm server in any candidate region)	Allocator error or timeout from the orchestrator	Teams are formed but the match can’t start	Teams returned to the pool with preserved wait credit; repeated failures alert on-call and trigger orchestrator warm-pool scale-up
Player fails to ready-up or disconnects during reservation	Ready-ack timeout in AwaitReadyCheck	Match can’t start with the full 10 players	Remaining players’ tickets requeue with wait time preserved; the disconnecting player receives a short matchmaking cooldown to discourage queue-dodging
Latency probe data is stale or missing for a player	Sample timestamp checked against the 30s freshness requirement	A match could route to a poor region for that one player	Falls back to the geoip_fallback estimate, flagged low-confidence, and weighted down in the quality score rather than trusted at face value
Clock skew between matchmaker workers	NTP drift monitors flag divergent timestamps across workers	Bracket expansion radius miscalculated; SLO metrics become unreliable	queued_at_ms is stamped once, centrally, by the Gateway at ticket ingestion, not independently by each worker, so downstream skew can’t corrupt the wait calculation

Comparison of Approaches

Approach	Queue Wait Behavior	Complexity	Failure Mode	Best Fit
FIFO / arrival-order matching	Fast and bounded, but ignores skill entirely	Low	Produces skill-mismatched stomps constantly, no fairness signal at all	Casual, unranked modes where fairness matters far less than instant play
Fixed narrow skill window, no expansion	Unbounded in thin brackets; a ticket can wait forever	Low to moderate	Off-peak or niche-bracket tickets never match, violating any wait SLO	Small, dense populations where every bracket always has enough players online
Latency-only / nearest-region lobby	Fast and low-lag, but ignores skill entirely	Low	Skill mismatches as severe as FIFO, just latency-optimized ones	LAN parties or small communities where the whole population is already skill-similar
Weighted quality-score with time-based bracket expansion (this design)	Bounded by construction; degrades gracefully instead of stalling	High (shard topology, quality tuning, party handling)	Requires careful tuning of growth rates and the quality floor; misconfiguration is a real operational risk	Large, latency-sensitive, skill-diverse populations, the common case for a ranked online multiplayer game
Central open lobby / join-in-progress, no formal matchmaking	Effectively zero wait, but zero fairness or latency control	Very low	Extreme skill and latency variance within the same lobby	Battle royale or sandbox modes where team balance matters far less than raw population density

For a ranked, skill-sensitive, latency-sensitive game at hundreds of thousands of concurrent players, the weighted quality-score approach with time-based bracket expansion is the only one of these that actually satisfies a wait-time SLO without simply abandoning fairness or playability as a goal. FIFO and latency-only lobbies are the right call when fairness genuinely doesn’t matter (a casual or LAN-scale mode), and a fixed narrow skill window only works if the population is dense enough everywhere, all the time, that expansion is never actually needed, which is rarely true once a game has enough regions and enough skill spread to be interesting. The complexity cost of the quality-score approach is real, but it’s the complexity of tuning a small number of well-understood constants against real telemetry, not the complexity of an unbounded, unpredictable failure mode.

Key Takeaways

• A single monotonic constraint (search radius only grows) is what turns “match quality” into a bounded guarantee: because a ticket’s window never shrinks, its search space provably reaches a fully open state at a calculable time, which is what actually protects the wait-time SLO.
• ELO’s local, two-number update is what lets rating changes scale: no global leaderboard recomputation is needed because a player’s new rating depends only on their own rating and the opponent’s average, making every update an independent, parallel write.
• The matchmaking pool has to be organized by region and coarse skill bucket, not by a single global structure: sharding is what keeps a search to O(log N + M) instead of a full scan, and it’s what isolates a hot bracket’s load from every unrelated bracket.
• Parties need synergy and variance adjustments, not a raw average: a naive average rating for a party is the single most common cause of one-sided stomps in real matchmakers, because it hides coordination advantage and internal skill spread behind a comfortable-looking number.
• Match quality only becomes a meaningful single number after normalization: comparing raw ELO points, milliseconds, and seconds directly is comparing incompatible units, and any fixed weight over un-normalized signals is arbitrary.
• Bounding worst-case wait and bounding worst-case quality are different, and only one of them can be promised unconditionally: bracket expansion guarantees the first; it can only degrade gracefully on the second, on purpose.
• Server allocation should optimize for the worst latency in the match, not the average: a great average with one unplayable outlier is still a broken match for that one player.
• A ready-check before allocation, with wait time preserved on failure, keeps a single flaky connection from costing the other nine players their place in line.

The counter-intuitive lesson here is that the hardest part of this system isn’t the algorithm most engineers assume it is. ELO, the part with the most textbook coverage, is genuinely the simplest piece: two numbers, one formula, a fully local update. The actual hard problem is admitting, in code, that a “good enough” match found now is sometimes worth more than a theoretically perfect match that might never arrive, and building a principled, monotonic, fully explainable mechanism for making that tradeoff automatically, millions of times a day, instead of leaving it to an ad hoc timeout somewhere in the codebase.

Frequently Asked Questions

Q: Why not use a fixed skill window instead of one that expands over time?

A: A fixed window only works if every skill bracket, in every region, at every hour, always has enough concurrent players to fill a match within your wait SLO, which is essentially never true once a game has any meaningful regional or off-peak variance. A ticket in a thin bracket with a fixed window simply waits forever, which isn’t a wait SLO violation, it’s a permanently broken promise to that player. Expansion trades some match quality for a hard guarantee that every ticket’s search space eventually becomes large enough to be satisfied.

Q: Why not use a purely latency-based, nearest-region matching approach and skip ELO entirely?

A: Latency-only matching produces the exact same fairness problem FIFO does, just optimized along a different axis; two players can be perfectly close in ping and wildly mismatched in skill, producing the same one-sided stomps. Skill and latency are solving genuinely different problems, playability versus fairness, and collapsing them into one signal throws away one of the two things players actually care about.

Q: Why not use a more sophisticated rating system like Glicko-2 or TrueSkill instead of classic ELO?

A: Glicko-2 and TrueSkill both model rating uncertainty explicitly (a confidence interval, not just a point estimate) and handle team games and partial information more natively, which is a real accuracy advantage, especially early in a player’s history. The tradeoff is complexity: both require tracking and updating a variance term per player, not just a rating, and TrueSkill’s factor-graph based updates are meaningfully more expensive to compute and reason about than a closed-form logistic formula. Classic ELO with a dynamic K-factor gets most of the practical benefit (fast convergence for new players, stability for established ones) at a fraction of the implementation and operational complexity, which is why it’s still the right default unless rating accuracy is the single most important product metric.

Q: How do you actually prevent party stacking abuse, where a coordinated group repeatedly queues against weaker solo players?

A: The synergy bump in party_effective_rating makes a party’s effective ELO higher than its raw average, so it’s matched against tougher opposition than a naive average would produce. On top of that, the Assembly Engine prefers party-versus-party matches over party-versus-solo whenever the population allows it, only falling back to mixed matches once a ticket’s wait time has grown enough that insisting on a same-size opponent would itself breach the SLO.

Q: How is the queue wait time SLO actually measured and enforced operationally, rather than just designed on paper?

A: Every ticket lifecycle event (queued, radius expanded, matched, cancelled, timed out) writes a row into ticket_audit with a timestamp, keyed by (region, game_mode, elo_bucket). A monitoring job continuously computes p95 and p99 wait time from that table per bucket, and an SLO breach in any one bucket pages on-call and can trigger a temporary, automated tightening of MAX_ELO_RADIUS growth caps for adjacent healthy buckets to absorb overflow, or a manual review of whether that bucket’s population has genuinely dropped below a sustainable threshold.

Q: What happens when an entire region’s game server fleet goes down?

A: New tickets destined for that region get rerouted to the next-nearest healthy region as part of their initial candidate cluster list, which costs those players some latency but keeps them matchable. In-progress matches already running on the degraded fleet are unaffected by this reroute; only new allocations are redirected. The Allocator’s BestCluster selection naturally deprioritizes a region with no available warm servers, since a region that can’t allocate returns ErrNoServerAvailable and is excluded from the candidate set entirely.

Interview Questions

Q: Design the search structure for a matchmaking pool that has to support tens of thousands of in-flight tickets with a search happening on a sub-second tick for each one.

Expected depth: Discuss why a single global sorted structure creates contention and unnecessary scan cost, and how sharding by region plus a coarse skill bucket turns a search into an O(log N + M) range query. Cover what happens when a search radius spans multiple buckets (fan-out across adjacent shards), and how bucket width becomes a tunable knob for hot brackets versus cold ones.

Q: Walk through how you’d design the mechanism that guarantees no player waits longer than a fixed SLO, even in a skill bracket with very few online players.

Expected depth: Explain time-based bracket expansion as a monotonic, non-decreasing function of wait time, covering both the skill radius and the latency ceiling, and why monotonicity is what makes the guarantee provable rather than just probable. Discuss the tradeoff this makes explicit: bounding worst-case wait time necessarily means giving up a guarantee on worst-case match quality, and why those two guarantees can’t both be made unconditionally.

Q: A player reports being matched into a wildly unbalanced game where their team was clearly much weaker. How would you investigate this in a system using ELO plus a quality-score matcher?

Expected depth: Walk through likely causes in order: the match may have formed near the fully-open end of bracket expansion (check ticket_audit wait time for that match), a party may have been under-rated due to a variance-penalty miscalculation, a quality floor that was recently loosened without replaying historical data, or a cold-start player still on a geoip_fallback latency estimate skewing the candidate pool. Discuss what telemetry is needed: quality_score and avg_skill_delta stored on the matches row, and each participant’s wait_time_ms and pre_match_elo.

Q: How would you extend party matchmaking to fairly handle a party of 5 queuing against a mix of parties and solo players, without either starving the party’s wait time or producing an unfair match?

Expected depth: Cover treating each party as an indivisible unit in the pool search and the snake-draft team assembly, the effective-rating adjustment (synergy bump plus variance penalty) that accounts for coordination advantage, and the soft preference for party-versus-party matches that only relaxes to party-versus-solo once wait time grows past a threshold. Discuss the tension between fairness (finding an equally-sized party opponent) and wait time (a thin population may not have one), and how that maps onto the same bracket-expansion machinery used for skill.

Q: The dedicated game server fleet in one region degrades mid-peak. Design the failover behavior for in-flight tickets and matches already assembled but not yet allocated a server.

Expected depth: Distinguish between tickets still searching (reroute their candidate cluster list to the next-nearest healthy region going forward), matches already formed but stuck in ready-check or allocation (return them to the pool with preserved wait time rather than dropping them), and matches already running on a healthy server in that region (unaffected, no action needed). Cover how BestCluster naturally excludes a region reporting ErrNoServerAvailable, and the operational tradeoff of a fully automated failover versus one gated behind a health threshold to avoid flapping during a brief, recoverable blip.