Picture a community corkboard in an office break room, where anyone can pin a flyer and anyone walking past can stick a thumbs-up or thumbs-down note next to it. A flyer pinned an hour ago with twelve thumbs-up probably deserves eye-level placement over a flyer from six days ago with fifteen thumbs-up, because the older one already had a hundred and forty-four hours to collect votes while the newer one earned its votes in a fraction of that time. But a flyer pinned thirty seconds ago with a single thumbs-up should not leapfrog a flyer from two hours ago sitting at eighty net positive votes just because it is newer. Deciding what belongs at eye level, right now, without physically walking over and re-pinning every flyer in the room every time one more sticky note gets added, is the entire problem a ranking algorithm exists to solve.

Now scale that corkboard to a platform where roughly 700,000 posts land every day across millions of active communities, and where those posts collectively receive on the order of 140 million votes a day, a sustained rate near 1,600 votes per second that bursts past 25,000 votes per second during a major live event, a viral thread, or a coordinated brigade. Readers expect a subreddit’s front page to render in well under 200ms at the 99th percentile, and they expect it to feel alive: a post that was buried an hour ago should be able to climb back toward the top if it suddenly catches fire, and a post from yesterday should keep sliding down even if nobody touches it again. Getting this right is not a matter of taste, it is a matter of physics: with millions of live posts and tens of thousands of votes arriving every second, whatever mechanism decides “what shows first” has to do a vanishingly small amount of work per vote, or it collapses under its own denominator.

The naive fix, sort every post in a subreddit by net upvotes at read time, fails on two fronts at once. First, ranking by raw net score treats a post with 1 upvote and 0 downvotes as “100% positive” and ranks it above a post with 950 upvotes and 60 downvotes, even though the second post has vastly more evidence behind its popularity and the first is one enthusiastic click away from being a coin flip. Second, sorting on every page load means re-deriving the order of every post in a subreddit’s history each time anyone requests a feed, which is fine for a subreddit with a few hundred posts and catastrophic for one with tens of millions. The opposite naive fix, freeze a post’s score forever the moment it is created, solves the recompute cost but breaks freshness completely: a post from last year with a huge net score would permanently outrank everything posted today, because nothing is pulling old content back down as time passes.

We need to solve for three things simultaneously: a scoring function that turns raw vote counts into a statistically defensible confidence signal instead of a raw tally, a decay mechanism that lets time erode a post’s rank continuously without a background job rewriting every row in the corpus, and an update path that touches only the single post that just received a vote, in roughly constant time, rather than re-deriving an entire subreddit’s order from scratch.

Requirements and Constraints

Functional Requirements

• Accept upvote, downvote, and vote-retraction events for a post, keyed by (post_id, user_id), so a user’s vote counts exactly once and can flip direction without double-counting
• Maintain three distinct orderings per subreddit: hot (time-decayed relevance), top (period-scoped confidence score), and new (pure chronological order)
• Combine raw ups and downs into a Wilson score lower-bound confidence interval, so posts with few votes cannot outrank posts with far more evidence at a similar or better ratio
• Apply a continuous time-decay term to the hot score so a post’s rank keeps falling as it ages, even if it receives no further votes at all
• Compute a separate controversy score per post to power a “controversial” sort and to help flag statistically unusual voting patterns
• Allow each subreddit to customize its ranking behavior (decay half-life, Wilson confidence level, weight given to comments and awards) through configuration, not a code deploy
• Update a post’s score incrementally in response to a single vote, touching only that post’s row and index entries, never resorting the rest of the feed
• Tolerate a bounded amount of score staleness for decay-only drift (no new vote), while guaranteeing a fresh vote is reflected in the index within a tight latency bound

Non-Functional Requirements

• Scale: roughly 700,000 new posts/day, 140,000,000 votes/day (about 1,600 votes/sec sustained, bursting to 25,000 votes/sec during major events), millions of active subreddits, with a “hot” working set of roughly 50 million actively-ranked posts (last 30 days) sitting on top of a multi-billion post historical corpus queried mainly through top: all time
• Latency: vote ingestion acknowledgment p99 under 80ms; a vote’s effect on the hot index visible within 500ms p99; a paginated feed page (25 posts, hot/top/new) render p99 under 150ms
• Score staleness tolerance: a hot score is allowed to drift up to 60 seconds stale relative to pure time decay when no new vote has occurred, but must reflect an actual vote within 500ms
• Durability: no vote may be silently lost; the vote log is durable and replayable from a fixed offset
• Availability: the feed read path targets 99.95% uptime; the vote write path targets 99.9%, and is allowed to degrade to “accepted, ranking delayed” before it is allowed to reject votes outright
• Consistency: the ranking index is eventually consistent by design; the underlying ups/downs counters for a single post are strongly consistent per post row

Constraints and Assumptions

• Comment content and comment-thread ranking are out of scope; this system ranks posts only, and treats comment_count as an input signal, not something it owns
• Awarding and gilding are treated as an input weight, not a payment or inventory system; we assume award events arrive as a simple count delta
• Moderation removal and ban enforcement happen upstream; this design assumes a removed flag is already set correctly on a post before it reaches the ranking layer
• User authentication and one-vote-per-user eligibility are solved by an existing auth system; this design assumes a valid, deduplicated user_id is attached to every vote
• Cross-subreddit personalization (a user’s individually tailored home feed, based on subscriptions and past behavior) is out of scope; this design covers subreddit-level and network-level (r/all-style) ranking only

High-Level Architecture

The system has six major components: the Vote Ingestion Gateway, the Vote Log and Dedupe Store, the Score Update Engine, the Subreddit Ranking Config Service, the Ranking Index Store, and the Feed Assembly Service.

The Vote Ingestion Gateway is the front door for every upvote, downvote, and retraction: it validates the request and treats every vote as idempotent on (post_id, user_id). The Vote Log and Dedupe Store is the durable, replayable record of every vote, and the source of truth if any downstream index ever needs to be rebuilt. The Score Update Engine is the core of the whole design: it applies a vote’s delta to a post’s ups/downs counters and recomputes that one post’s Wilson score, decayed hot score, and controversy score in roughly constant time, without touching any other post. A companion Controversy and Anti-Spam Processor watches the same vote stream for statistically unusual patterns, both to compute the controversy score and to catch coordinated manipulation. The Subreddit Ranking Config Service holds per-community tuning (decay half-life, Wilson confidence, comment and award weights) so one subreddit’s culture doesn’t have to rank identically to another’s. The Ranking Index Store holds three sorted indices per subreddit, one each for hot, top, and new, and is the only thing a feed read ever touches. The Feed Assembly Service paginates those indices, merges across subreddit shards for network-wide views, and is what actually answers “give me the next 25 posts.”

A vote’s life looks like this: a reader clicks upvote, the gateway checks the vote against its (post_id, user_id) key so a network retry never double-counts, and on success the vote is appended to the durable log. The Score Update Engine picks up that event, applies the delta to the post’s counters, and recomputes the Wilson score, hot score, and controversy score for that single post, using per-subreddit parameters pulled from the config service. The updated scores are written into the Ranking Index Store’s hot, top, and controversial sorted sets with a single atomic operation, and the Feed Assembly Service’s next read for that subreddit picks up the new ordering immediately. Independent of any vote at all, a bucketed decay-refresh sweep periodically re-touches recently active posts so the hot score’s time-decay term keeps eroding correctly even for posts that stop receiving votes, bounded by the staleness tolerance from the requirements above.

Component Deep Dives

The Vote Ingestion Gateway

This component’s job is to make sure the exact same vote, retried because of a flaky mobile connection or a client that never received its response, is never counted twice.

The obvious mistake is trusting a client to send a vote exactly once. Mobile clients drop connections mid-request constantly, and a client that times out waiting for an acknowledgment has no way to know whether the server actually processed the vote before the connection died, so a correct client retries, and the server has to treat that retry as harmless. We key every vote by (post_id, user_id) and treat the operation as an upsert of direction, not an append-only insert of an event.

-- Vote upsert: a user's vote on a post is a single row that can change
-- direction, never a growing list of separate vote events for the same pair
INSERT INTO votes (post_id, user_id, direction, cast_at)
VALUES ($1, $2, $3, NOW())
ON CONFLICT (post_id, user_id)
DO UPDATE SET direction = EXCLUDED.direction, cast_at = EXCLUDED.cast_at
RETURNING (
    SELECT direction FROM votes WHERE post_id = $1 AND user_id = $2
) AS previous_direction;

Think of this like a light switch rather than a tally counter: flipping the same switch to “on” five times in a row leaves the room exactly as lit as flipping it once, because the switch has one state, not a running count of flips. What breaks without this: a shopper on a spotty connection taps upvote once, the request goes through, the acknowledgment is lost, the client retries, and a naive counter-increment design counts two upvotes for one click, quietly inflating a post’s score for votes that were never actually cast twice by a human.

The Score Update Engine

This component’s job is to take a single vote’s effect and turn it into an updated Wilson score, hot score, and controversy score for the one post that vote touched, without looking at any other post.

The non-obvious part is that “incremental” does not mean “approximate.” Every quantity this engine computes, a Wilson lower bound, a decayed hot score, a controversy measure, is a pure function of a post’s current ups, downs, and created_at, which means recomputing it from scratch for one post is already O(1) work. The expensive thing was never the math, it was the idea of recomputing every post’s math on every read. Once you accept that a single post’s score only needs to change when that post’s inputs change, the “incremental” update falls out for free: apply the vote’s delta to ups/downs with an atomic counter operation, then recompute that post’s three scores using the exact same formulas a full recompute would use, and write the result into the index.

The two formulas doing the actual scoring work are the Wilson score lower bound and the time-decayed hot score:

import math
from datetime import datetime, timezone

EPOCH = datetime(2005, 12, 8, tzinfo=timezone.utc)  # a fixed reference point; only relative time matters

def wilson_score_lower_bound(ups: int, downs: int, z: float = 1.96) -> float:
    """
    Wilson score lower bound for a Bernoulli proportion (fraction of
    upvotes), used instead of a raw ups/(ups+downs) ratio so posts with a
    handful of votes cannot outrank posts with far more supporting evidence.
    z=1.96 corresponds to a 95% confidence level.
    """
    n = ups + downs
    if n == 0:
        return 0.0

    phat = ups / n
    z2 = z * z

    numerator = phat + z2 / (2 * n) - z * math.sqrt((phat * (1 - phat) + z2 / (4 * n)) / n)
    denominator = 1 + z2 / n

    return numerator / denominator

def _sign(x: float) -> int:
    return (x > 0) - (x < 0)

def hot_score_from_epoch(ups: int, downs: int, created_at_epoch: float, decay_half_life_seconds: int = 45000) -> float:
    """
    Time-decayed hot ranking score. The log10 term rewards vote magnitude
    with diminishing returns, while the linear time term steadily erodes
    rank as a post ages, independent of whether it receives further votes.
    decay_half_life_seconds is a per-subreddit tunable pulled from
    subreddit_ranking_config: a smaller value makes hot decay faster.
    """
    net_score = ups - downs
    order = math.log10(max(abs(net_score), 1))
    sign = _sign(net_score)
    reference_epoch = EPOCH.timestamp()
    return round(sign * order + (created_at_epoch - reference_epoch) / decay_half_life_seconds, 7)

With those two building blocks in place, applying a vote is just: bump the counters, recompute, write:

# Incremental score update triggered by a single vote event.
# Demonstrates: only the voted-on post's counters and index entries are
# touched, never a resort of the subreddit's feed.
def apply_vote_and_rescore(redis_client, post_id: str, subreddit_id: str,
                            delta_ups: int, delta_downs: int,
                            created_at_epoch: float, decay_half_life_seconds: int,
                            wilson_z: float) -> dict:
    post_key = f"post:{post_id}"

    pipe = redis_client.pipeline()
    pipe.hincrby(post_key, "ups", delta_ups)
    pipe.hincrby(post_key, "downs", delta_downs)
    ups, downs = pipe.execute()[-2:]

    hot = hot_score_from_epoch(ups, downs, created_at_epoch, decay_half_life_seconds)
    wilson = wilson_score_lower_bound(ups, downs, wilson_z)
    controversy = controversy_score(ups, downs)

    redis_client.hset(post_key, mapping={
        "hot_score": hot, "wilson_score": wilson, "controversy_score": controversy,
    })

    write_pipe = redis_client.pipeline()
    write_pipe.zadd(f"rank:{subreddit_id}:hot", {post_id: hot})
    write_pipe.zadd(f"rank:{subreddit_id}:top:day", {post_id: wilson})
    write_pipe.zadd(f"rank:{subreddit_id}:controversial", {post_id: controversy})
    write_pipe.execute()

    return {"ups": ups, "downs": downs, "hot_score": hot, "wilson_score": wilson, "controversy_score": controversy}

The analogy is a scoreboard operator at a live sporting event: when a team scores, the operator does not recalculate the entire season’s standings, they update that one game’s number on that one scoreboard, and the standings simply reflect whatever the scoreboards currently say the next time anyone looks. What breaks if this were done as a full recompute instead: at 25,000 votes/sec during a peak event, resorting even a modest 100,000-post subreddit on every single vote would require on the order of a million comparisons per vote, a workload no single scoring path can sustain, while touching one post’s fields is a handful of arithmetic operations regardless of how large the subreddit is.

The Controversy and Anti-Spam Processor

This component’s job is to compute a controversy score that rewards posts with heavy, closely-split voting, and to watch the same vote stream for patterns that look like coordinated manipulation rather than organic disagreement.

A smart engineer’s first assumption is usually that “controversial” just means “close to 50/50,” but a post with 2 upvotes and 2 downvotes and a post with 20,000 upvotes and 20,000 downvotes both have an identical 1.0 balance ratio, and only one of them is actually a controversial post in any meaningful sense. Reddit’s own published formula, still in wide use, multiplies the balance ratio by the total vote magnitude so that both a close split and a large volume are required before something counts as genuinely controversial.

def controversy_score(ups: int, downs: int) -> float:
    """
    Rewards posts with a large volume of votes split close to 50/50, rather
    than posts that are simply unpopular. A 2-2 split and a 20000-20000 split
    both have balance 1.0, but the second is far more controversial because
    of the magnitude term.
    """
    if ups <= 0 or downs <= 0:
        return 0.0

    magnitude = ups + downs
    balance = min(ups, downs) / max(ups, downs)
    return magnitude ** balance

The same processor watches vote velocity per post, bucketed by account age and by the network neighborhood of voting accounts (many brand-new accounts voting the same direction on the same post inside a short window), and flags that pattern for review rather than trusting every vote equally. Think of this like a casino’s floor pit boss watching for a group of players making suspiciously correlated bets at the same table: any single bet looks legitimate on its own, and it is only the correlated pattern across many accounts, arriving in a tight time window, that reveals coordinated behavior. What breaks without this: a coordinated brigade of a few hundred throwaway accounts can push an otherwise unremarkable post to the top of a subreddit’s hot feed in minutes, and a scoring system with no anti-manipulation signal has no way to distinguish that from a post genuinely resonating with tens of thousands of organic readers.

The Subreddit Ranking Config Service

This component’s job is to let each subreddit tune how aggressively its hot feed decays and how much weight comments or awards contribute, without anyone touching the scoring code itself.

The naive assumption is that one global decay curve and one global Wilson confidence level should work for every community, but a subreddit built around fast-moving live discussion (a sports game thread, a breaking news megathread) wants old posts to fall out of hot within hours, while a subreddit built around long-form writing or slow-burn discussion wants posts to stay visible for days. A single hardcoded decay constant forces every subreddit into the same rhythm regardless of what its actual audience wants.

CREATE TABLE subreddit_ranking_config (
    subreddit_id             BIGINT PRIMARY KEY,
    decay_half_life_seconds  INT NOT NULL DEFAULT 45000,
    wilson_confidence_z      DOUBLE PRECISION NOT NULL DEFAULT 1.96,
    comment_weight           DOUBLE PRECISION NOT NULL DEFAULT 0.15,
    award_weight             DOUBLE PRECISION NOT NULL DEFAULT 0.05,
    controversy_enabled      BOOLEAN NOT NULL DEFAULT TRUE,
    updated_at               TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

The Score Update Engine reads this row (cached aggressively, since it changes rarely and is read on every single vote) and threads its values straight into the Wilson and decay formulas covered below. Think of it like a thermostat schedule that each household sets independently in an apartment building sharing one boiler: the underlying heating mechanism is identical for everyone, but each household dials in the temperature curve that suits how they actually live. What would break if config were baked into code instead of data: every tuning change would require a full deploy, and a moderator team wanting to slow down decay during a slow news week would have no lever to pull without an engineer in the loop.

The Ranking Index Store and Feed Assembly Service

The Ranking Index Store’s job is to hold three separate, always-sorted orderings per subreddit, hot, top, and new, so a feed read is a single range query against an already-sorted structure rather than a sort performed at read time. The Feed Assembly Service’s job is to turn that range query into a paginated response, including the scatter-gather merge needed for network-wide views like an r/all-style aggregation across many subreddits’ shards at once.

The three feeds are not three views over the same number, they are three genuinely different scoring functions serving three different intents. hot uses the time-decayed score, favoring recent momentum. top uses the Wilson lower bound over a chosen time window (day, week, month, year, all-time), favoring statistically well-supported quality regardless of recency. new uses nothing but created_at, favoring pure chronological order for readers who want to see everything as it lands, unranked. Maintaining all three as separate sorted sets, updated by the same incremental write, means a reader flipping between sort tabs gets three consistent, independently-correct orderings instead of one order approximated three different ways at read time.

rank:{subreddit_id}:hot                -> ZSET member=post_id score=hot_score
rank:{subreddit_id}:top:day             -> ZSET member=post_id score=wilson_score
rank:{subreddit_id}:top:all             -> ZSET member=post_id score=wilson_score
rank:{subreddit_id}:new                 -> ZSET member=post_id score=created_at_epoch
rank:{subreddit_id}:controversial       -> ZSET member=post_id score=controversy_score

Reading a page of hot is a single ZREVRANGE over the subreddit’s own sorted set, an O(log N + page_size) operation regardless of how many posts that subreddit has ever received. Network-wide views like r/all cannot avoid touching multiple subreddit shards, so the Feed Assembly Service performs a bounded scatter-gather: it asks each shard for its own top-K candidates, then merges those small candidate lists client-side, which keeps the fan-out cost proportional to the number of shards, not the number of posts.

Data Model

The data model spans four entities: posts, votes, per-subreddit ranking configuration, and an append-only score audit log used for reconciliation and abuse investigation.

-- Posts: the durable row a vote ultimately updates. hot_score, wilson_score,
-- and controversy_score are denormalized here so the ranking index can be
-- rebuilt from this table if a Redis shard is ever lost outright.
CREATE TABLE posts (
    post_id             BIGINT PRIMARY KEY,
    subreddit_id        BIGINT NOT NULL,
    author_id           BIGINT NOT NULL,
    created_at          TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    ups                 INT NOT NULL DEFAULT 0 CHECK (ups >= 0),
    downs               INT NOT NULL DEFAULT 0 CHECK (downs >= 0),
    comment_count       INT NOT NULL DEFAULT 0 CHECK (comment_count >= 0),
    award_count         INT NOT NULL DEFAULT 0 CHECK (award_count >= 0),
    removed             BOOLEAN NOT NULL DEFAULT FALSE,
    hot_score           DOUBLE PRECISION NOT NULL DEFAULT 0,
    wilson_score        DOUBLE PRECISION NOT NULL DEFAULT 0,
    controversy_score   DOUBLE PRECISION NOT NULL DEFAULT 0,
    score_updated_at    TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX ON posts (subreddit_id, hot_score DESC) WHERE removed = FALSE;
CREATE INDEX ON posts (subreddit_id, wilson_score DESC) WHERE removed = FALSE;
CREATE INDEX ON posts (subreddit_id, created_at DESC) WHERE removed = FALSE;

-- Votes: one row per (post, user) pair, upserted rather than appended, so a
-- vote is idempotent and a direction change never double-counts
CREATE TABLE votes (
    post_id     BIGINT NOT NULL REFERENCES posts(post_id),
    user_id     BIGINT NOT NULL,
    direction   SMALLINT NOT NULL CHECK (direction IN (-1, 0, 1)),
    cast_at     TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    PRIMARY KEY (post_id, user_id)
);

CREATE INDEX ON votes (post_id);

-- Subreddit-level ranking customization, read on every score recomputation
CREATE TABLE subreddit_ranking_config (
    subreddit_id             BIGINT PRIMARY KEY,
    decay_half_life_seconds  INT NOT NULL DEFAULT 45000,
    wilson_confidence_z      DOUBLE PRECISION NOT NULL DEFAULT 1.96,
    comment_weight           DOUBLE PRECISION NOT NULL DEFAULT 0.15,
    award_weight             DOUBLE PRECISION NOT NULL DEFAULT 0.05,
    controversy_enabled      BOOLEAN NOT NULL DEFAULT TRUE,
    updated_at               TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Append-only audit log backing reconciliation and abuse investigation
CREATE TABLE score_audit_log (
    id                  BIGSERIAL PRIMARY KEY,
    post_id             BIGINT NOT NULL,
    trigger             TEXT NOT NULL CHECK (trigger IN ('vote', 'decay_refresh', 'moderation')),
    ups                 INT NOT NULL,
    downs               INT NOT NULL,
    hot_score           DOUBLE PRECISION NOT NULL,
    wilson_score        DOUBLE PRECISION NOT NULL,
    computed_at         TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX ON score_audit_log (post_id, computed_at DESC);

posts and votes are sharded by hashing subreddit_id, not post_id alone. Almost every read in this system, a hot page, a top page, a new page, is scoped to one subreddit, so co-locating a subreddit’s posts and votes on the same shard means the overwhelmingly common query pattern never has to fan out across the cluster. The rare network-wide view is the one query type that accepts a scatter-gather cost in exchange for keeping the common case single-shard.

A post is born with ups = 0, downs = 0, and its hot_score seeded from just its creation time. Every subsequent vote runs the idempotency check, appends to the vote log, and triggers the Score Update Engine to touch that one row: ups/downs change atomically, and hot_score, wilson_score, and controversy_score are all recomputed from the new values and written back, both to the durable posts row and to the in-memory ranking index. Independent of any vote, the decay-refresh sweep periodically re-touches hot_score for recently active posts so time decay is reflected even during a lull in voting, keeping drift bounded by the staleness tolerance rather than letting it grow unbounded between votes.

Key Algorithms and Protocols

Wilson Score Lower Bound Confidence Interval

Covered in the Score Update Engine section above: wilson_score_lower_bound(ups, downs, z) computes the lower bound of a confidence interval around the true upvote proportion, rather than the raw ratio ups / (ups + downs). A raw ratio treats a post with 1 upvote and 0 downvotes as “perfect,” ranking it above a post with 950 upvotes and 50 downvotes despite the second post having overwhelmingly more evidence of genuine quality. With n = 0 the function returns 0.0 rather than dividing by zero, and as n grows the lower bound converges toward the true ratio phat: a brand-new post with one vote gets a conservative, low score, and a post with thousands of votes at the same ratio gets a score close to its actual ratio.

Time-Decay Hot Ranking Formula

Covered in the Score Update Engine section above: hot_score_from_epoch(ups, downs, created_at_epoch, decay_half_life_seconds) combines a log-scale vote-magnitude term with a linear time-decay term. The hot score needs two properties that raw vote count alone cannot give it: diminishing returns on vote magnitude (going from 10 to 20 net votes should matter more than going from 10,000 to 10,010), and continuous erosion by elapsed time so a post’s rank keeps falling even with zero further activity. The sign * order term means a post with net -50 votes decays symmetrically to one with net +50 votes, just downward instead of upward, which correctly separates “actively disliked” from “simply unvoted.” The elapsed-time term divided by decay_half_life_seconds is what actually pulls old posts down: two posts with identical vote counts but different ages will always rank by age, because the time term dominates once vote counts stop changing.

Controversy Score

Covered in the Controversy and Anti-Spam Processor section above: magnitude ** balance, where balance = min(ups, downs) / max(ups, downs) and magnitude = ups + downs, ensures a post needs both a large vote volume and a near-even split to register as genuinely controversial, in constant time per post.

Incremental Score Update

Covered in the Score Update Engine section above: a vote’s delta is applied to ups/downs with an atomic HINCRBY-style operation, and the resulting three scores are recomputed directly from the new counter values, an O(1) operation per post regardless of the size of the subreddit that post belongs to.

Score Staleness Refresh (Bucketed Decay Sweep)

Because the hot score’s time term erodes continuously, a post’s rank drifts relative to newer posts even when it receives zero further votes, and a purely vote-triggered update would leave that drift uncorrected until the next vote happens to arrive, possibly hours later. We tolerate bounded staleness by running an active sweep that re-touches recently active posts on a schedule, more frequently for very fresh posts and less frequently as they age, rather than scanning the entire corpus on every tick.

// Decay refresh sweeper: periodically re-touches hot_score for recently
// active posts so time decay is reflected even without a new vote event.
// Posts are bucketed by age so freshly posted content refreshes far more
// often than posts already hours old, bounding staleness without scanning
// the entire corpus on every tick.
package decay

import "time"

type AgeBucket struct {
	MaxAge   time.Duration
	Interval time.Duration
}

var buckets = []AgeBucket{
	{MaxAge: 30 * time.Minute, Interval: 15 * time.Second},
	{MaxAge: 6 * time.Hour, Interval: 60 * time.Second},
	{MaxAge: 48 * time.Hour, Interval: 5 * time.Minute},
}

func (s *Sweeper) refreshDueBuckets(now time.Time) {
	for _, b := range buckets {
		if now.Sub(s.lastRun[b.Interval]) < b.Interval {
			continue
		}
		s.refreshPostsNewerThan(b.MaxAge)
		s.lastRun[b.Interval] = now
	}
}

Posts older than the oldest bucket’s MaxAge are excluded from the sweep entirely: their hot rank has already fallen so far behind fresher content that further decay refresh changes nothing observable, and re-touching them would spend cycles on rows nobody is reading through the hot sort anyway.

Scaling and Performance

The Ranking Index Store and the Score Update Engine sit under the most sustained load, since every one of the roughly 140 million daily votes touches both, and the system scales primarily by sharding both by subreddit_id and by handling the rare viral post as a special case rather than optimizing the common path for it.

Sharding by subreddit_id keeps the overwhelmingly common query pattern, “give me this one subreddit’s feed,” entirely single-shard, and it also keeps per-subreddit ranking config colocated with the data it configures. The exception is a single post that goes viral inside one subreddit: tens of thousands of votes a second landing on one rank:{subreddit_id}:hot sorted set turns that one key into a serialization point, since even an atomic per-key operation still executes one at a time against a single key. We handle this the same way a hot inventory counter would be handled: once a post’s vote rate crosses a threshold, its contribution is split across several virtual sub-counters, summed together only when a read actually needs the true total.

Capacity Estimation:

Given:
  - ~700,000 posts/day, ~140,000,000 votes/day
  - Peak vote rate: 25,000 votes/sec (major event burst)
  - Hot working set: ~50,000,000 actively-ranked posts (last 30 days)
  - Average vote record: ~40 bytes; average post row: ~250 bytes

Vote log write throughput:
  25,000 votes/sec * 40 bytes = ~1 MB/sec sustained write bandwidth,
  comfortably within range for a partitioned, append-only log

Score recompute throughput:
  Each vote triggers one O(1) recompute plus 3 ZADD writes (hot, top, one
  controversy) = 25,000 * 4 = ~100,000 index operations/sec at peak,
  spread across subreddit shards, not concentrated on one node

Ranking index memory footprint:
  50,000,000 hot-window posts * 3 sorted-set entries * ~80 bytes/entry
  = ~12 GB, shardable across a modest Redis cluster

Decay-refresh sweep volume:
  Posts under 30 minutes old refresh every 15s: roughly 700,000 posts/day
  / (24*3600/15) intervals implies a rolling few thousand posts touched
  per 15-second tick, a small fraction of total vote-driven writes

Feed read throughput:
  Assume 10x more feed page reads than votes cast: 25,000 * 10 =
  250,000 reads/sec at peak, each a single ZREVRANGE against one
  subreddit's own shard, O(log N + page_size) per read

The dominant bottleneck is never aggregate throughput, the numbers above are modest by distributed systems standards, it is the handful of individual posts that receive disproportionate concurrent vote traffic during a viral spike. Sharding by subreddit spreads ordinary load evenly across the cluster, and virtual sub-counter splitting handles the small number of keys where even a perfectly atomic single-key operation becomes a throughput ceiling.

Failure Modes and Recovery

Failure	Detection	Impact	Recovery
Score Update Engine consumer lag builds up on the vote stream	Growing consumer-group lag metric on the vote topic	Hot feed reflects vote counts that are stale beyond the 500ms SLA; readers see outdated ordering	Autoscale consumer instances; the durable vote log allows replay from the last committed offset with no data loss
Ranking Index shard for a subreddit goes down	Connection failures, replica-promotion alarm on that shard	Reads for that subreddit’s hot/top/new fail or serve from a stale replica during failover	A replica promotes and resumes serving; if the shard’s data is lost entirely, the index is rebuilt from posts.hot_score/wilson_score denormalized columns, not a full vote-log replay
Coordinated vote brigade (bot storm) targets one post	Velocity anomaly on vote rate correlated with newly created or low-karma accounts	Score is artificially inflated or deflated, ranking manipulated away from organic reader sentiment	Anti-spam processor quarantines flagged votes pending review; score is recomputed excluding quarantined votes, original votes remain in the durable log for audit
Viral post causes hot-key contention on one subreddit’s rank:hot ZSET	p99 latency spike isolated to one Redis key or shard, CPU saturation on that node	Vote acknowledgment slows for that post; feed reads for the whole subreddit can degrade if the shard is shared	Post’s contribution is split into virtual sub-counters once its vote rate crosses a threshold, spreading load across multiple physical keys
Clock skew across regions affects the decay calculation	A post’s hot_score drifts inconsistently depending on which region computed it	Same post ranks differently depending on which region served the read	Decay is computed against created_at, a value fixed once at post creation and never recomputed against a different node’s local clock; the reference epoch is a constant, not “now”
Duplicate vote event replay (client retry or stream redelivery)	Unique-constraint upsert on (post_id, user_id) absorbs the retry with no error	Without idempotency, a naive counter-increment design would double-count a single click	The votes table stores current direction per pair rather than an appended event count, so replay is a no-op or a corrective update, never a double increment

Comparison of Approaches

Approach	Latency	Complexity	Failure mode	Best fit
Recompute-on-read (sort every post at query time)	Degrades sharply as subreddit size grows; fine at hundreds of posts, unusable at millions	Low	A single popular subreddit’s feed page becomes slower for everyone as its history grows, with no upper bound	Small forums or a prototype, never a subreddit with meaningful history
Fully static score, frozen at post creation, never updated	Fastest possible read, no recompute ever	Low	Old high-scoring posts permanently dominate; the feed stops feeling current within hours	Archival or read-only historical views where freshness is explicitly not a goal
Periodic batch recompute (cron job re-scores the whole corpus every N minutes)	Read latency is fine; freshness is bounded by the batch interval, often minutes	Medium	A viral post’s score lags the batch window; a brigade has minutes to do damage before the next run	Systems where minute-scale staleness is acceptable and vote volume is too low to justify a streaming pipeline
Incremental push-based update with a bounded staleness decay sweep (this design)	Sub-second update on vote, decay drift bounded and small	Medium-high	Requires careful staleness-budget tuning and index/source-of-truth reconciliation tooling	Any high-vote-volume, multi-community platform needing both freshness and sub-second read latency
Client-side re-rank of only the currently visible page	Cheapest possible server cost; reordering happens in the browser	Low on the server, but pushes correctness risk to the client	Different readers can see different orders for the same feed at the same moment, undermining trust in “what’s actually hot”	Low-stakes personalization layered on top of a server-computed base order, never as the primary ranking mechanism

We would pick the incremental push-based design with a bounded staleness decay sweep for any platform operating at more than a trivial vote volume, because it is the only approach here that keeps both halves of the requirement, sub-second freshness on real votes and a feed read that never has to sort anything at request time. The alternatives each give up one side of that tradeoff: recompute-on-read trades away latency as the corpus grows, a fully static score trades away freshness entirely, batch recompute trades away timeliness during exactly the events (viral spikes, brigades) where timeliness matters most, and client-side re-ranking trades away a single, trustworthy notion of “hot” shared across all readers.

Key Takeaways

• A ranking score should be a pure function of one post’s own state. Wilson score, hot score, and controversy score all depend only on ups, downs, and created_at for that single post, which is exactly what makes an incremental, per-post update correct rather than approximate.
• Wilson score lower bound beats a raw net-vote ratio by requiring both a good ratio and enough votes to be statistically confident in it, so a post with one lucky upvote can never outrank one with thousands of votes at a similar ratio.
• Time decay has to be a closed-form function of created_at, not a value rewritten on a schedule for every post. That is what lets a post’s rank keep falling with age without a background job touching every row in the corpus.
• Hot, top, and new are three different scoring functions, not three views of one number. Maintaining all three as separately-updated sorted indices keeps every sort tab correct without approximating one from another.
• Subreddit-level customization has to be data the scoring engine reads per request, not a constant baked into the formula, because moderation teams need to tune decay and weighting on a timescale no code deploy can match.
• Controversy is a function of both split and magnitude, not split alone. A small, evenly-split vote count and a massive, evenly-split vote count are not equally controversial, and the formula has to reflect that.
• Bounded staleness is a deliberate budget, not an accident. Tolerating up to 60 seconds of pure decay drift while holding vote-driven updates to 500ms spends slack exactly where a human reader cannot perceive it.
• A hot key survives a viral spike the same way a hot inventory row does: split it into virtual sub-counters, summed on read, rather than trying to make one key handle unbounded concurrent writes.

The counter-intuitive lesson is that the hardest-sounding part of this system, correctly ranking millions of posts under tens of thousands of votes per second, is solved by formulas that are individually almost trivial, a handful of arithmetic operations per post. The actual engineering difficulty is everywhere else: making sure every formula only ever needs one post’s own data so updates stay O(1), deciding exactly how much staleness is acceptable and where to spend that budget, and building the tooling that keeps a fast, denormalized index honest against its slower, durable source of truth.

Frequently Asked Questions

Q: Why not just sort posts by raw net upvotes (ups - downs) instead of a Wilson score?

A: Because net votes conflate volume with confidence: a post with 1 upvote and 0 downvotes has a “perfect” net score of 1, and a naive ratio-based sort can rank it above a post with 950 upvotes and 60 downvotes, even though the second post has far more evidence behind its actual popularity. The Wilson lower bound explicitly accounts for sample size, so a post needs sustained support at scale, not a lucky handful of early votes, to rank highly.

Q: Why not just recompute every post’s rank on every feed read instead of maintaining a live index?

A: Recompute-on-read is O(n log n) in the size of a subreddit’s post history on every single page view, which is fine for a subreddit with a few hundred posts and unworkable for one with millions. Maintaining hot/top/new as continuously-updated sorted sets turns a read into a single range query, independent of how large the subreddit’s total history is, at the cost of doing a small amount of extra work at vote time instead.

Q: How do you keep a viral post’s vote traffic from becoming a single point of contention?

A: Once a post’s vote rate crosses a configured threshold, its score contribution is split across a fixed number of virtual sub-counters distributed across different shards, and incoming votes are routed to one at random. Reads sum all sub-counters together to get the true total. This is the same technique used to handle a single hot inventory row under flash-sale-level contention: no single key, however atomic its operations, has unlimited throughput.

Q: What stops a coordinated brigade of new accounts from gaming a post’s rank?

A: The anti-spam processor watches vote velocity correlated with account age and voting-account overlap across posts, and flags patterns that look coordinated rather than organic. Flagged votes are quarantined, excluded from scoring pending review, but retained in the durable vote log rather than deleted, so a moderation decision can be reversed without losing the underlying record.

Q: Why maintain a decay-refresh sweep at all if hot_score is just a function of created_at that can be computed at read time?

A: Computing hot_score purely at read time is a valid alternative for a single feed page, but this design also denormalizes hot_score onto the durable posts row and into the fast index for range queries and cross-shard merges, and those copies would silently go stale if nothing ever re-touched them between votes. The bucketed sweep keeps that denormalized copy inside the staleness budget without scanning the entire corpus, refreshing only the recently active slice where decay actually changes visible rank.

Q: How do subreddit-specific ranking parameters avoid becoming a performance bottleneck if they are read on every vote?

A: subreddit_ranking_config rows change extremely rarely relative to how often they are read, so they are cached aggressively at the Score Update Engine, invalidated only when a moderator actually changes a value. The read pattern is effectively “read a rarely-changing row millions of times,” which is exactly the shape a local or near cache handles well.

Interview Questions

Q: Walk through what happens, end to end, when a post that has been sitting quietly for three hours suddenly receives 5,000 upvotes in one minute.

Expected depth: Cover the idempotency check per vote, the Score Update Engine recomputing hot/wilson/controversy for that one post per vote, the threshold that triggers virtual sub-counter splitting once vote velocity crosses it, and why the rest of the subreddit’s feed is completely unaffected by this one post’s surge.

Q: A subreddit wants its hot feed to feel much faster-moving than the platform default, with posts falling out of prominence within a couple of hours instead of half a day. How would you support that without touching the scoring code?

Expected depth: Discuss decay_half_life_seconds as a per-subreddit config value read by the Score Update Engine, why a smaller half-life makes the linear time term dominate the hot score sooner, and how caching that rarely-changing config value avoids adding read latency to every vote.

Q: Why is a Wilson score lower bound a better choice than a raw upvote ratio, and what would you do differently if you needed the ranking to be strictly non-manipulable by a large number of low-quality accounts each casting one vote?

Expected depth: Explain the confidence-interval intuition, discuss the z parameter’s effect on how conservative the bound is with low vote counts, and connect to the anti-spam processor’s job of discounting or quarantining votes from patterns that look coordinated rather than organic before they ever reach the Wilson calculation.

Q: The system needs to support a global, cross-subreddit “all” feed showing the best posts network-wide. What changes from the per-subreddit design, and where are the new costs?

Expected depth: Cover the scatter-gather merge across subreddit shards, why each shard only needs to contribute its own top-K rather than its entire sorted set, the latency cost of fanning out to every shard versus a single-shard read, and why this view is deliberately treated as the exception rather than optimized as the common case.

Q: A moderator asks why a post from two days ago with 40,000 upvotes still outranks a post from this morning with 2,000 upvotes on hot, and wants to know if that is a bug. How do you explain and, if needed, tune the behavior?

Expected depth: Walk through the log-scale vote term versus the linear time term in the hot formula, explain that this is expected behavior given the current decay half-life, and discuss adjusting decay_half_life_seconds for that subreddit as the correct lever rather than treating it as a defect in the scoring formula itself.