A coat check at a busy theater works on a simple contract: you hand over your coat, you get a numbered ticket, and for the rest of the night that ticket is proof of who you are to the cleanup desk. The attendant does not re-verify your identity on every visit to the counter between acts. They check the ticket number against a rack position and hand the coat back in milliseconds. The system only breaks down if the rack burns down, the ticket gets lost, or two people are handed the same number.

A session store is the coat check for the web. Every authenticated request a user makes - load a page, call an API, open a websocket - needs to answer “who is this and are they still logged in” without re-running a full authentication flow each time. At 50 million active sessions, that lookup has to happen in under 5 milliseconds, millions of times per second, and it has to survive losing a server without a single user noticing they were logged out.

The naive approach is to store sessions in the same relational database that holds your user accounts. That works at 10,000 sessions. At 50 million it does not, for three compounding reasons. First, session reads vastly outnumber writes (every page view is a read, only login and logout are writes), so you are paying disk-backed query latency for what should be a microsecond-scale lookup. Second, sessions expire constantly and continuously - every active user’s session needs its TTL refreshed on each request if you support sliding expiry - which turns your database into an update-heavy workload it was never built for. Third, a relational database’s failover takes seconds, and during that window every authenticated request in your system fails.

The forces in tension are speed, durability, and availability during failure. We want session reads fast enough to be invisible (sub-5ms), we want a node crash to be invisible to the user (no forced logout), and we want a security model that does not leave 50 million long-lived tokens sitting around as a standing attack surface. We need to solve for in-memory speed, replicated durability, and graceful failover simultaneously, and each of those pulls the design in a different direction.

Requirements and Constraints

Functional Requirements

• Create a session on successful login and return an opaque session token to the client
• Validate a session token and return the associated session data (user ID, roles, metadata) on every authenticated request
• Support sliding expiry: each successful read extends the session’s time-to-live
• Support explicit logout (single session) and global logout (all sessions for a user)
• Rotate session tokens periodically without forcing re-authentication
• Survive the failure of any single storage node without invalidating active sessions

Non-Functional Requirements

• Read latency: p99 under 5ms for a session lookup
• Write latency: p99 under 10ms for session create, update, and TTL refresh
• Scale: 50 million concurrent active sessions, sustained read throughput of 800,000 ops/sec, peak 1.5M ops/sec
• Availability: 99.99% for the session validation path - this sits in front of every authenticated API call
• Durability: a node failure must not log out an active user; acceptable to lose sessions that were created in the last 100ms before a crash, not acceptable to lose anything older
• Session size: average 1.2KB per session (user ID, roles, device metadata, feature flags)

Constraints and Assumptions

• We assume a stateless application tier - app servers do not pin users to specific instances based on session location
• We are not designing the authentication flow itself (password checks, OAuth handshakes) - only what happens after a user is authenticated
• We assume client devices can store an opaque token (cookie or Authorization header) but never see the session’s internal data structure
• Long-term session analytics (login history, audit trails) are out of scope - those live in a separate event log, not the hot session store
• We assume a single logical datacenter region per deployment; cross-region session replication is discussed but not the primary design target

High-Level Architecture

The system has five major components: the Session Gateway, Redis Cluster (the session store itself), the Replication Layer, the Token Service, and an Async Write-Behind Log for durability beyond memory.

A request arrives at the Session Gateway carrying a session token in a cookie or header. The Gateway computes which shard owns that token using consistent hashing, and issues a GET against the owning Redis Cluster node. If the session exists and has not expired, the Gateway extends its TTL in the same round trip (sliding expiry) and forwards the request to the application tier with the deserialized session payload attached. If the node holding the shard is unreachable, the Gateway transparently retries against that shard’s replica, which Redis Cluster promotes to primary within seconds of detecting the failure.

The Redis Cluster is the source of truth for live session reads and writes. It is sharded across 16,384 hash slots, each slot owned by one primary node with at least one replica. The Replication Layer streams every write from a primary to its replicas asynchronously, and synchronously enough (via WAIT) for sessions that require strict durability, like the session created at login. The Token Service issues, validates the format of, and rotates session tokens - it does not own session data, only the cryptographic and routing logic baked into the token itself. The Async Write-Behind Log ships a sampled, compacted copy of session lifecycle events (create, rotate, revoke) to a durable log for audit and for warming a cold cluster after a full regional failure.

Data flows in a tight loop: client sends token, Gateway resolves shard, Redis answers in microseconds, Gateway refreshes TTL, response returns. The only component on this hot path that does meaningful computation is the Gateway’s shard resolution, which is a single hash operation.

Component Deep Dives

The Session Gateway

The Session Gateway’s job is to turn an opaque token into a shard lookup and a sub-5ms answer, without becoming a stateful bottleneck itself.

A smart engineer’s first instinct is to put a local cache in front of Redis inside the Gateway to shave off the network hop entirely. That works until you have more than one Gateway instance, which you always do at this scale - now you have N caches that can each be stale by a different amount, and a logout on one Gateway instance does not invalidate the cached copy on the other nineteen. We do use a very short-lived (under 1 second) local cache for read-heavy hot sessions, but we treat Redis as the only durable source of truth and never let a Gateway serve a write without going through it.

The Gateway is stateless and horizontally scaled behind a load balancer. Each instance independently computes the target shard for a token using the same hash function Redis Cluster uses internally (CRC16 of the token’s hash tag, modulo 16,384), so it can route directly to the owning node instead of bouncing through a Redis Cluster redirect on every request.

# Session Gateway: resolve shard and fetch session with sliding TTL refresh
# Demonstrates: CRC16 slot resolution, single round-trip read + TTL extend
import zlib

TOTAL_SLOTS = 16384
SESSION_TTL_SECONDS = 1800  # 30 minutes sliding window

def hash_slot(token: str) -> int:
    # Redis Cluster CRC16 slot algorithm, honoring {hashtag} if present
    if "{" in token and "}" in token:
        start = token.index("{") + 1
        end = token.index("}", start)
        if end > start:
            token = token[start:end]
    return zlib.crc32(token.encode()) % TOTAL_SLOTS

class SessionGateway:
    def __init__(self, cluster_client, shard_map):
        self.cluster = cluster_client
        self.shard_map = shard_map  # slot range -> node endpoint

    def get_session(self, token: str) -> dict | None:
        slot = hash_slot(token)
        node = self.shard_map.node_for_slot(slot)
        # Lua script: GET + sliding TTL refresh in one atomic round trip
        result = self.cluster.eval(
            REFRESH_AND_GET_SCRIPT,
            keys=[f"session:{token}"],
            args=[SESSION_TTL_SECONDS],
            node=node,
        )
        if result is None:
            return None
        return self.deserialize(result)

REFRESH_AND_GET_SCRIPT = """
local val = redis.call('GET', KEYS[1])
if val then
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return val
"""

Running the read and the TTL refresh as a single Lua script avoids a second round trip and guarantees the two operations are atomic - no other client can delete the key between the GET and the EXPIRE. Without this, a logout racing a read could refresh a session’s TTL a few milliseconds after it was supposed to be deleted, resurrecting a session that should be dead.

Redis Cluster Sharding

Redis Cluster’s job is to hold 50 million session records in memory, spread across enough nodes that no single node is a bottleneck, while keeping every related read within one hop.

The naive sharding strategy is hash(token) mod N where N is the node count. The problem with that scheme is obvious the moment you add or remove a node: nearly every key remaps to a different node, which means nearly every session becomes unreachable at its old location simultaneously. Redis Cluster instead uses a fixed 16,384-slot keyspace. Each node owns a contiguous or scattered range of slots, and only the slots that move during a resharding operation are affected - the other 16,000+ slots are untouched.

Think of it like a coat check with 16,384 numbered hooks distributed across several rooms, rather than one hook per coat-check counter. Adding a new room just means moving a few hundred hooks worth of coats into it; nobody else’s coat changes hooks.

# Provisioning a Redis Cluster sized for 50M sessions at 1.2KB average
# Demonstrates: cluster create, slot allocation, replica assignment

# 12 primary nodes, 1 replica each = 24 nodes total
# Each primary holds ~1,365 slots and ~4.2M sessions (~5GB resident)
redis-cli --cluster create \
  10.0.1.1:6379 10.0.1.2:6379 10.0.1.3:6379 \
  10.0.1.4:6379 10.0.1.5:6379 10.0.1.6:6379 \
  10.0.1.7:6379 10.0.1.8:6379 10.0.1.9:6379 \
  10.0.1.10:6379 10.0.1.11:6379 10.0.1.12:6379 \
  10.0.2.1:6379 10.0.2.2:6379 10.0.2.3:6379 \
  10.0.2.4:6379 10.0.2.5:6379 10.0.2.6:6379 \
  10.0.2.7:6379 10.0.2.8:6379 10.0.2.9:6379 \
  10.0.2.10:6379 10.0.2.11:6379 10.0.2.12:6379 \
  --cluster-replicas 1

# Resharding: move 200 slots from node A to node B without downtime
redis-cli --cluster reshard 10.0.1.1:6379 \
  --cluster-from <node-a-id> \
  --cluster-to <node-b-id> \
  --cluster-slots 200 \
  --cluster-yes

Choosing the shard key matters as much as the algorithm. We key sessions by the token itself (session:{token}), not by user ID, because a user’s session lookups are always single-key point reads - there is no query pattern that needs all of one user’s sessions co-located on one node. If we did need that (for “log out all my devices”), we would use a hash tag like session:{user:42}:device1 to force co-location of a user’s sessions on one shard, at the cost of creating hot shards for power users with many devices.

Replication and Durability

This component’s job is to make sure a session created on one node is not lost the instant that node dies.

The instinctive design is synchronous replication on every write: the primary does not acknowledge a SET until every replica has it too. That guarantees zero data loss, but it also means your write latency is now bounded by your slowest replica, and a network blip to one replica stalls every session write in the cluster. At 50 million sessions with constant TTL refreshes counting as writes, that is not survivable.

We use asynchronous replication for routine TTL refreshes (the vast majority of write volume) and synchronous replication, via Redis’s WAIT, only for session creation and explicit logout - the two events where losing the write actually matters to the user. A lost TTL refresh just means the session expires a few seconds earlier than the sliding window intended, which is harmless. A lost session creation means a user who just logged in gets logged out a second later, which is a visibly broken experience.

# Session creation with synchronous replication acknowledgment
# Demonstrates: WAIT for replica ack, bounded wait with fallback
import redis

SESSION_TTL_SECONDS = 1800
REPLICA_ACK_TIMEOUT_MS = 8  # fail open rather than blow the 10ms write budget

def create_session(client: redis.Redis, token: str, payload: bytes) -> bool:
    pipe = client.pipeline(transaction=True)
    pipe.set(f"session:{token}", payload, ex=SESSION_TTL_SECONDS, nx=True)
    results = pipe.execute()
    if not results[0]:
        return False  # token collision, extremely rare with 128-bit tokens

    # Require at least 1 replica to acknowledge within 8ms.
    # If it times out, the write already happened on the primary -
    # we log a durability warning but do not fail the login.
    acked = client.execute_command("WAIT", 1, REPLICA_ACK_TIMEOUT_MS)
    if acked < 1:
        log_durability_warning(token)
    return True

The 8ms timeout on WAIT is a deliberate choice: we would rather accept a small risk window of replica lag than blow our 10ms write SLA waiting for acknowledgment. The replica almost always acks well under a millisecond on the same rack; the timeout exists to bound the worst case, not to be hit routinely.

Sticky Sessions vs Stateless Lookup

This component’s job is to decide where a request’s session data has to be found, and whether the app server handling the request needs to “remember” anything about the user between requests.

Sticky sessions - routing a user’s traffic to the same application server every time via a load balancer cookie - feel like the simpler design at first. The app server can hold session state in local memory, lookups are free, and there is no Redis at all. The catch is that sticky sessions turn every app server into a single point of failure for the users pinned to it. Restart that server for a deploy and every user pinned to it loses their session unless you build a complicated drain-and-migrate dance. Scaling becomes uneven too: a server that happens to be sticky-routed to many active users runs hot while others sit idle, and the load balancer cannot rebalance without breaking sessions.

We use the opposite design: stateless application servers and a centralized, replicated session store that any app server can query. Any request can land on any app server, because the session lookup is a fast, location-independent call to the Redis Cluster rather than a lookup in local memory. This is the design that makes horizontal autoscaling, rolling deploys, and graceful failover all simple, at the cost of paying a network round trip on every request that sticky sessions would have avoided. At our 5ms budget, that round trip is affordable; it would not have been affordable at a 0.5ms budget, which is the regime where sticky sessions or client-side JWTs start looking attractive again.

Graceful Failover

This component’s job is to make sure that when a Redis node dies mid-flight, the next request from an affected user succeeds anyway, without the user knowing anything happened.

The naive expectation is that failover is just “promote the replica and move on.” In practice the dangerous window is the few seconds between the primary disappearing and the cluster agreeing a replica should take over. Redis Cluster nodes gossip liveness via a heartbeat protocol; a node is marked PFAIL (possibly failing) by any node that cannot reach it, and only promoted to fully FAIL once a majority of primaries agree. Only after that consensus does an eligible replica run for election and get promoted. During that handshake, requests to the affected slots return errors or time out.

# redis.conf settings that control failover timing
cluster-node-timeout 5000        # ms before a node is marked PFAIL
cluster-replica-validity-factor 10
cluster-replica-no-failover no   # replicas are allowed to auto-promote
cluster-require-full-coverage no # serve requests for healthy slots even if some are down

The Gateway is the layer that hides this window from the end user. It treats any connection error or timeout from a Redis node as a signal to retry once against that shard’s known replica address before propagating an error upward, and it implements a short exponential backoff with jitter so a thundering herd of retries does not itself overwhelm the newly promoted primary.

# Gateway-side failover handling: retry against replica, then degrade gracefully
# Demonstrates: bounded retry, replica fallback, circuit breaker on repeated failure
import time
import random

class FailoverAwareClient:
    def __init__(self, primary_pool, replica_pool, max_retries=2):
        self.primary_pool = primary_pool
        self.replica_pool = replica_pool
        self.max_retries = max_retries
        self.breaker_open_until = {}

    def get(self, key: str, node_id: str) -> bytes | None:
        if self.breaker_open_until.get(node_id, 0) > time.monotonic():
            return self._read_replica(key, node_id)

        for attempt in range(self.max_retries + 1):
            try:
                return self.primary_pool.get(node_id).get(key)
            except (ConnectionError, TimeoutError):
                if attempt == self.max_retries:
                    self.breaker_open_until[node_id] = time.monotonic() + 10
                    return self._read_replica(key, node_id)
                backoff = (0.005 * (2 ** attempt)) + random.uniform(0, 0.003)
                time.sleep(backoff)
        return None

    def _read_replica(self, key: str, node_id: str) -> bytes | None:
        try:
            return self.replica_pool.get(node_id).get(key)
        except (ConnectionError, TimeoutError):
            return None  # both primary and replica down - rare, total slot outage

What makes this graceful rather than just “fast failing” is that the session data itself already exists on the replica before the failure, because of the replication design from the previous section. Failover does not need to reconstruct anything; it only needs to redirect traffic to data that was already there.

Token Rotation

This component’s job is to limit how long any single session token is valuable to an attacker who manages to steal one, without forcing the user to log in again.

A session token that lives unchanged for the lifetime of a login is a standing liability: if it leaks via a logged URL, a browser extension, or an XSS bug, it remains valid for as long as the session does. The fix is to rotate the token periodically while keeping the underlying session continuous from the user’s point of view.

# Token rotation: issue a new token, keep old one valid briefly for in-flight requests
# Demonstrates: dual-token grace window, atomic swap via Lua
import secrets
import time

ROTATION_INTERVAL_SECONDS = 900   # rotate every 15 minutes of activity
GRACE_PERIOD_SECONDS = 30         # old token stays valid this long after rotation

def maybe_rotate_token(redis_client, old_token: str, session_payload: dict) -> str:
    last_rotated = session_payload.get("rotated_at", 0)
    if time.time() - last_rotated < ROTATION_INTERVAL_SECONDS:
        return old_token  # not due yet

    new_token = secrets.token_urlsafe(32)
    session_payload["rotated_at"] = time.time()
    session_payload["prior_token"] = old_token

    rotate_script = """
    local payload = redis.call('GET', KEYS[1])
    if not payload then return 0 end
    redis.call('SET', KEYS[2], ARGV[1], 'EX', ARGV[2])
    redis.call('EXPIRE', KEYS[1], ARGV[3])
    return 1
    """
    redis_client.eval(
        rotate_script,
        keys=[f"session:{old_token}", f"session:{new_token}"],
        args=[serialize(session_payload), 1800, GRACE_PERIOD_SECONDS],
    )
    return new_token

The 30-second grace period matters because a client might have an in-flight request carrying the old token at the exact moment of rotation - a mobile app with a slow network, a retried request, a second browser tab. Killing the old token immediately would intermittently log out users mid-session for no visible reason. Letting it die naturally after a short overlap window closes the attack surface quickly while staying invisible to legitimate traffic.

Data Model

The session record itself is a single serialized blob per token, plus a small set of secondary structures for token rotation and multi-device logout.

-- Reference schema for the durable audit copy (Postgres), not the hot path.
-- The hot path never touches this table directly - it is populated
-- asynchronously from the write-behind log.
CREATE TABLE session_audit_log (
    id              BIGSERIAL   PRIMARY KEY,
    session_token_hash TEXT     NOT NULL,  -- SHA-256, never store raw tokens
    user_id         BIGINT      NOT NULL,
    event_type      TEXT        NOT NULL
        CHECK (event_type IN ('create', 'rotate', 'refresh_skip', 'revoke', 'expire')),
    device_id       TEXT,
    ip_address      INET,
    user_agent      TEXT,
    occurred_at     TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX ON session_audit_log (user_id, occurred_at DESC);
CREATE INDEX ON session_audit_log (session_token_hash, occurred_at DESC);

-- Per-user device registry for "log out all devices" support.
-- Small table, queried only on explicit global logout - not the hot path.
CREATE TABLE user_devices (
    user_id         BIGINT      NOT NULL,
    device_id       TEXT        NOT NULL,
    current_token_hash TEXT     NOT NULL,
    last_seen_at    TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    PRIMARY KEY (user_id, device_id)
);

The live record in Redis is not relational at all - it is a binary-packed payload stored under a single key:

# Session payload structure stored as the Redis value (msgpack-encoded)
# Demonstrates: compact session schema, ~1.2KB average serialized size
SESSION_SCHEMA_EXAMPLE = {
    "user_id": 8839201,
    "roles": ["member", "beta_tester"],
    "device_id": "ios-7f3a9c",
    "created_at": 1751270400,
    "rotated_at": 1751271300,
    "prior_token": None,           # set briefly during rotation grace window
    "feature_flags": {"new_checkout": True},
    "ip_hash": "9f86d081884c7d65",  # hashed, not raw IP - privacy + smaller diffing
}

We key by session:{token} and deliberately do not index by user_id inside Redis - cross-referencing all of a user’s sessions for “log out everywhere” is a low-frequency operation that goes through user_devices in Postgres instead, which is the appropriate tool for a secondary index that is not on the hot path. Partitioning is by token hash slot, as covered in the sharding section, which keeps every individual session lookup to one shard regardless of cluster size.

A session record’s life cycle moves through four states: created (login, synchronous replication), active (read-heavy, sliding TTL extension on every hit), rotated (token swapped, old token in grace period), and terminated (explicit logout, global logout, or natural TTL expiry). The diagram traces a record through each of these transitions and which component drives each one.

Key Algorithms and Protocols

Consistent Hashing for Slot Assignment

Redis Cluster’s slot model is a specific, simplified form of consistent hashing: instead of placing nodes directly on a hash ring, it fixes 16,384 virtual slots on the ring and assigns contiguous or scattered ranges of those slots to physical nodes. Resharding moves slot ownership, not individual keys directly - the keys inside a moved slot follow their slot.

# Slot assignment and rebalance planning
# Demonstrates: even slot distribution across N nodes, minimal-movement rebalance
TOTAL_SLOTS = 16384

def assign_slots(node_ids: list[str]) -> dict[str, range]:
    n = len(node_ids)
    base = TOTAL_SLOTS // n
    remainder = TOTAL_SLOTS % n
    assignments = {}
    start = 0
    for i, node_id in enumerate(node_ids):
        count = base + (1 if i < remainder else 0)
        assignments[node_id] = range(start, start + count)
        start += count
    return assignments

def slots_to_migrate(old_assignment: dict, new_assignment: dict) -> list[tuple]:
    # Only slots whose owner changed need migration - this is the property
    # that makes adding a 13th node to a 12-node cluster cheap.
    old_owner = {slot: node for node, r in old_assignment.items() for slot in r}
    new_owner = {slot: node for node, r in new_assignment.items() for slot in r}
    moves = []
    for slot, new_node in new_owner.items():
        if old_owner.get(slot) != new_node:
            moves.append((slot, old_owner.get(slot), new_node))
    return moves

Adding a 13th node to a 12-node, 16,384-slot cluster moves roughly 16384 / 13 ≈ 1,260 slots, about 8% of the keyspace, instead of the ~92% of keys that a naive hash mod N scheme would remap. Time complexity for slot lookup is O(1) (array index by slot number); space overhead is the slot map itself, a few hundred KB per node regardless of cluster size.

Sliding TTL Expiry

Sliding expiry means a session’s lifetime is measured from its most recent activity, not its creation time. Every successful read pushes the expiry window forward.

# Sliding TTL semantics, explicit edge case handling
# Demonstrates: TTL extension cap, idle vs absolute expiry combination
MAX_SESSION_LIFETIME_SECONDS = 86400 * 7   # absolute cap: 7 days, even if always active
IDLE_TTL_SECONDS = 1800                     # 30 min sliding window

def should_extend_ttl(session: dict, now: float) -> bool:
    absolute_age = now - session["created_at"]
    if absolute_age >= MAX_SESSION_LIFETIME_SECONDS:
        return False  # hard cap reached - force re-authentication regardless of activity
    return True

def compute_next_ttl(session: dict, now: float) -> int:
    absolute_age = now - session["created_at"]
    remaining_to_cap = MAX_SESSION_LIFETIME_SECONDS - absolute_age
    return int(min(IDLE_TTL_SECONDS, remaining_to_cap))

A session with pure sliding expiry and no absolute cap can theoretically live forever as long as the user stays active, which is rarely the security posture you want. We combine sliding expiry (resets on activity, bounds idle abandonment) with an absolute maximum lifetime (forces periodic re-authentication regardless of activity). The Lua script shown earlier in the Gateway section implements the common case; the absolute cap check happens at session creation time by storing created_at once and never updating it on refresh.

Scaling and Performance

The system scales horizontally along two axes: more Redis Cluster shards to hold more sessions and absorb more write throughput, and more Gateway instances to absorb more request volume. The two scale independently because the Gateway is stateless.

Capacity Estimation:

Given:
  - 50,000,000 active sessions
  - 1.2 KB average serialized session size
  - 800,000 reads/sec sustained, 1,500,000 reads/sec peak
  - 15,000 writes/sec (logins + rotations + logouts combined)
  - 30 min sliding TTL window

Memory:
  Raw session data: 50,000,000 * 1.2 KB = 60 GB
  With Redis overhead (~40% for key metadata, expiry, hash table buckets): ~84 GB
  With 1 replica per shard (full duplication): ~168 GB total cluster memory

Throughput per node (r6g.2xlarge class, 8 vCPU):
  Single Redis node sustains ~120,000 simple GET ops/sec at sub-1ms local latency
  Required nodes for 1.5M peak reads/sec: 1,500,000 / 120,000 ≈ 13 primaries
  Round up with headroom: 16 primaries, 16 replicas = 32 nodes total

Network:
  Average response: 1.2 KB payload + ~200 bytes protocol overhead ≈ 1.4 KB
  Peak bandwidth: 1,500,000 * 1.4 KB = 2.1 GB/s aggregate read bandwidth
  Per-node bandwidth at 16 primaries: 2.1 GB/s / 16 ≈ 131 MB/s/node (comfortable on a 10Gbps NIC)

Sliding TTL refresh load:
  Each of the 800,000 sustained reads/sec triggers one EXPIRE call
  This is folded into the same round trip via the Lua script, not a separate op

The read path dominates by roughly 50:1 over the write path, which is the expected shape for a session store - users browse far more than they log in. We do not add a separate caching tier in front of Redis Cluster, because Redis already is the cache; adding another cache layer in front of an in-memory store mostly adds invalidation complexity without a meaningful latency win. The one exception is the Gateway’s sub-second local cache for extremely hot sessions (service accounts, bots making thousands of requests per second), which exists purely to protect a handful of shards from being disproportionately hammered by a small number of high-QPS callers.

Failure Modes and Recovery

Failure	Detection	Impact	Recovery
Single Redis primary crash	Cluster gossip marks node PFAIL then FAIL after majority agreement	Slots owned by that primary return errors for ~2-5 seconds	Replica auto-promotes via cluster election; Gateway retries against replica meanwhile
Network partition splits cluster	Nodes on minority side cannot reach majority of primaries	Minority-side nodes stop serving writes (cluster-require-full-coverage)	Majority side continues serving; minority rejoins and resyncs once partition heals
Replica falls behind on replication	Replication offset lag exceeds threshold, monitored via INFO replication	Failover to that replica would lose recent writes	Cluster prefers replicas with freshest offset for promotion; lagging replica is deprioritized
Gateway instance crash	Load balancer health check fails	In-flight requests to that instance fail	Load balancer routes new requests to healthy Gateway instances; client retries are idempotent reads
Token collision on creation	SETNX returns false	New session would silently overwrite an existing one	Generate with secrets.token_urlsafe(32) (effectively zero collision probability); retry with a new token if it ever happens
Clock skew across nodes	TTL expiry times computed inconsistently across replicas	A session might appear expired on one node and valid on another	Use Redis’s own internal expiry clock (PEXPIRE relative offsets, not absolute timestamps) so expiry is computed by each node from its own monotonic clock
Full cluster memory exhaustion	used_memory approaches maxmemory, eviction policy triggers	Sessions evicted before their TTL, forcing surprise re-logins	Capacity-plan with 30%+ headroom; alert at 70% memory; maxmemory-policy volatile-ttl evicts soonest-to-expire sessions first, never sessions without a TTL

Comparison of Approaches

Approach	Latency	Complexity	Failure Mode	Best Fit
Relational DB (Postgres) for sessions	10-30ms typical, worse under load	Low (reuses existing DB)	Failover takes seconds; write-heavy TTL refresh degrades whole DB	Low session volume (under ~100K), or sessions that must join against relational data anyway
Sticky sessions, in-memory on app server	Under 1ms (no network hop)	Low to build, high operationally	Node restart logs out pinned users; uneven load distribution	Small deployments where deploy-time logout is acceptable
Client-side JWT, no server-side store	Under 1ms (no lookup at all)	Medium (signing, key rotation)	Cannot revoke a single session before expiry without a denylist	Stateless APIs, mobile clients, short-lived tokens with low revocation need
Sharded Redis Cluster (this design)	Sub-5ms p99	High (cluster ops, failover tuning)	Single shard failure is brief and contained; whole-cluster failure is rare	High-volume, revocable, low-latency session needs - the common case for consumer web/mobile platforms
DynamoDB / managed NoSQL with TTL	5-15ms typical	Low operationally (managed), but TTL deletion is not instant (up to 48h delay in some systems)	Regional outage affects all sessions; TTL-based expiry is approximate, not exact	Teams wanting to avoid operating Redis themselves, willing to trade some latency and TTL precision

For a system with 50 million active sessions and a 5ms read SLA, a sharded Redis Cluster is the right call: it is the only option in this table that combines sub-5ms reads with precise, immediate TTL control and the ability to revoke a single session instantly. JWTs are tempting for their zero-lookup latency, but the inability to cheaply revoke a single compromised token before its natural expiry is a security gap that most consumer platforms are not willing to accept. The operational cost of running Redis Cluster correctly - failover tuning, resharding discipline, memory headroom - is the price of that combination of speed and control.

Key Takeaways

• Sliding TTL must be O(1): implementing it as a single EXPIRE call folded into the read path, not a read-modify-write of the session payload, is what keeps 800,000 reads/sec from becoming 800,000 read-modify-writes/sec.
• Slot-based sharding decouples rebalancing from lookup: Redis Cluster’s 16,384 fixed slots mean adding a node moves a small fraction of keys instead of remapping the whole keyspace, unlike naive hash mod N sharding.
• Not all writes need the same durability: synchronous replication for session creation and logout, asynchronous for routine TTL refreshes, cuts the volume of writes waiting on replica acknowledgment by over 95%.
• Stateless application servers plus a centralized session store beats sticky sessions at this scale, trading a network hop for simple horizontal scaling and painless rolling deploys.
• Graceful failover depends on data already being replicated before the failure - the Gateway’s retry-against-replica logic only works because the replica already has the data, not because failover somehow reconstructs it.
• Token rotation needs a grace period: killing the old token the instant a new one is issued breaks in-flight requests; a short overlap window closes the security gap without breaking legitimate traffic.
• Separate the hot path from the audit trail: the live session store optimizes purely for latency, while a separate asynchronously-populated audit log handles compliance and analytics queries that would otherwise slow down every login.
• Combine sliding and absolute expiry: pure sliding expiry can keep a session alive indefinitely as long as a user stays active, which is rarely the security posture you actually want.

The counter-intuitive lesson is that the hardest part of this system is not the data structure or even the sharding - it is deciding, write by write, how much durability each kind of write actually needs. Treating every write as equally precious is what makes naive session stores slow, and treating every write as equally disposable is what makes them unsafe. The entire design is a series of decisions about which writes get to skip the slow, safe path.

Frequently Asked Questions

Q: Why not just use a relational database with a well-indexed sessions table instead of Redis?

A: A relational database can absolutely serve session reads, but at 50 million sessions with 800,000 reads/sec and continuous TTL refreshes, you are asking a disk-oriented, transaction-oriented system to behave like an in-memory cache. Postgres can get you into the tens of milliseconds with good indexing and connection pooling, but the constant TTL update churn (every read potentially triggering a write) creates write amplification that a relational engine handles far worse than an in-memory key-value store built for exactly this access pattern. Redis is not “a database that happens to be fast” here, it is purpose-built for single-key point reads and writes at this volume.

Q: Why use opaque server-side tokens instead of self-contained JWTs and skip the lookup entirely?

A: JWTs eliminate the network hop, which is attractive, but they trade away instant revocation. If a JWT is valid for 30 minutes and a user’s account is compromised, you cannot invalidate that specific token without either waiting out its expiry or maintaining a denylist, at which point you have rebuilt a session store anyway, just for revocations instead of all sessions. For a consumer platform where “log out this device now” needs to actually mean now, an opaque token with a fast server-side lookup is the more honest design.

Q: How do you handle “log out of all devices” efficiently if sessions are not indexed by user ID in Redis?

A: We maintain a small secondary table (user_devices in Postgres) mapping each user to their active device tokens, updated on login and rotation. A global logout query is low-frequency, runs against Postgres, and then issues individual DEL commands against each affected token’s shard in Redis Cluster. This keeps the high-frequency hot path (single-token lookup) free of any secondary indexing overhead, at the cost of global logout being a slightly more expensive, multi-step operation - which is the right tradeoff given how rarely it happens compared to ordinary reads.

Q: What happens to in-flight requests during a Redis Cluster resharding operation?

A: Redis Cluster supports live resharding: a slot being migrated briefly exists in an IMPORTING state on the destination node and a MIGRATING state on the source. Requests for keys in that slot during the migration window get an ASK redirect telling the client to retry against the destination node. A well-behaved client (or our Gateway) honors that redirect transparently. The window where this matters is small - resharding moves slots in small batches specifically to keep any single slot’s migration window short.

Q: Why is the absolute session lifetime cap necessary if sliding TTL already expires inactive sessions?

A: Sliding TTL protects against an abandoned session lingering forever, but it does nothing for an attacker who has stolen a token and is actively using it to stay “active” indefinitely, since each fraudulent request resets the clock just like a legitimate one would. The absolute cap forces re-authentication on a fixed schedule regardless of activity, which bounds the maximum value of a stolen token even in the worst case where the attacker is indistinguishable from the legitimate user traffic-wise.

Q: How do you avoid a thundering herd of re-authentication requests when 50 million sessions all created around the same absolute-lifetime boundary expire simultaneously?

A: We deliberately add jitter (a few minutes of randomized offset) to the absolute lifetime cap at session creation time, so a cohort of users who logged in around the same event (a product launch, a marketing campaign) does not all hit their 7-day cap in the same narrow window. Without jitter, you get a synchronized wave of forced re-logins that looks identical to an authentication outage from a monitoring perspective.

Interview Questions

Q: Design the sharding strategy for a session store that needs to scale from 1 million to 50 million sessions without a “remap everything” event. What are the failure modes of getting this wrong?

Expected depth: Discuss consistent hashing versus Redis Cluster’s fixed 16,384-slot model, and why decoupling the rebalancing unit (a slot) from the lookup unit (a key) bounds the blast radius of adding or removing nodes. Cover the naive hash mod N failure mode explicitly: nearly all keys remap on any node count change. Discuss how clients discover slot ownership (MOVED/ASK redirects, client-side slot caching) and what happens when a client’s cached slot map is stale during a live resharding operation.

Q: A user reports they were logged out unexpectedly during normal browsing, with no explicit logout action. Walk through how you would debug this in a system with sliding TTL and Redis Cluster replication.

Expected depth: Walk through the candidate causes in order of likelihood: TTL refresh failing silently due to a Gateway bug, a failover promoting a replica that was lagging and missed the most recent TTL extension, a token rotation race where the old token’s grace period expired before the client picked up the new one, or memory pressure triggering eviction under volatile-ttl before natural expiry. Discuss what telemetry you would need (per-token refresh history, replication lag at time of failure, eviction logs) to distinguish between these.

Q: Your team wants to reduce session read latency below 5ms at the 99.9th percentile, not just the 99th. What would you investigate?

Expected depth: Discuss tail latency sources distinct from median latency: GC pauses on the Gateway or Redis itself, network jitter on the path between Gateway and Redis node, a small number of hot keys overloading one shard, and the cost of cross-AZ network hops if the Gateway and the owning Redis node are not co-located. Mention techniques like request hedging (firing a duplicate request to a replica after a short delay if the primary hasn’t responded) and the tradeoff of added load that hedging introduces.

Q: How would you extend this design to support session data replicated across two geographic regions for disaster recovery, and what consistency model would you choose?

Expected depth: Discuss the tension between strong cross-region consistency (which would blow the latency budget given inter-region RTT) and eventual consistency (which risks a user in region A having a session that region B does not yet know about during a regional failover). Cover options: active-passive with async cross-region replication and accepting a small data-loss window on regional failover, or active-active with conflict resolution favoring “most recent write wins” since sessions are mutable but not collaboratively edited. Mention that the audit log and the live session store likely need different cross-region strategies given their different durability requirements.

Q: Walk through what happens end-to-end, from both a correctness and a latency perspective, when a Redis primary holding 4 million sessions crashes during peak traffic.

Expected depth: Cover detection (gossip protocol, PFAIL to FAIL transition, timing controlled by cluster-node-timeout), the brief window where requests to those slots fail or timeout, replica election and promotion, and how the Gateway’s retry-against-replica logic bridges that window for the client. Discuss what is lost (writes that had not yet replicated, bounded by the synchronous WAIT design for session creation) versus what is preserved (everything that completed synchronous replication). Quantify the user-visible impact: a few seconds of elevated error rate for roughly 1/16th of all sessions (one shard’s worth out of sixteen primaries), not a global outage.