Think of Netflix’s content library as a warehouse full of goods that 280 million customers want delivered instantly. Now imagine you must pre-position those goods across 200+ local depots worldwide - before you know exactly what each customer will order. You have limited shelf space at each depot, a catalog of 15,000+ titles (each weighing up to a terabyte in all its bitrate variants), and a hard rule: the goods must arrive within 50ms of the customer pressing “play.” A centralized warehouse would fail immediately - the speed of light from a datacenter in Virginia to a user in Mumbai cannot be compressed by any software trick. Proximity to the user is the only physics-compliant answer.

Netflix serves approximately 15 petabytes of video data per day, with peak traffic exceeding 600 gigabits per second in North America alone. That scale forces a counter-intuitive architectural choice: Netflix does not rely primarily on commercial CDNs like Akamai or Cloudflare. Instead, it built its own CDN called Open Connect, placing purpose-built appliances (OCAs - Open Connect Appliances) directly inside Internet Service Provider networks. An OCA sitting inside a Comcast datacenter needs zero hops over the public internet to reach a Comcast subscriber - the video bytes travel through the ISP’s own internal routing. Netflix pays the ISP with bandwidth savings rather than cash; the ISP saves on transit costs; the subscriber gets sub-millisecond latency to the serving node.

The hard problem is not the serving layer itself - flash storage is fast and HTTP/2 byte-range streaming is mature. The hard problem is the cache population decision: which of the 15,000+ titles do you preload into each PoP tonight, given that you only have 100TB of flash storage and the catalog is 1 exabyte? Get this wrong and users see buffering. Get it too conservative and cache hit rates drop below 95%, flooding origin servers. The decision must account for time-of-day viewing patterns, local content preferences (Korean dramas dominate Seoul’s PoP; Telenovelas dominate Mexico City’s), upcoming release windows, and content that is trending on social media before it spikes in requests.

We need to solve for three things simultaneously: cache placement (what goes where, decided offline each night by the Cache Advisor), cache eviction (what gets bumped when new content must enter a full cache), and cache miss handling (how to serve a request for content not yet at the local PoP without slamming origin). The routing and consistency model that glues these together is consistent hashing, ensuring that the same title maps to the same PoP nodes regardless of the number of appliances in a cluster - so a cache miss from one appliance doesn’t trigger a redundant fetch from another.

Requirements and Constraints

Functional Requirements

• Serve video chunks (typically 2-10 second MPEG-TS or CMAF segments) with under 50ms time-to-first-byte at the edge
• Prefill PoP caches nightly based on predicted popularity per geographic region
• Handle cache miss by routing to a regional cluster (origin shield), not directly to S3 origin
• Evict cache entries based on a combined recency and frequency score, not pure LRU or pure LFU
• Invalidate stale cached content (e.g., after a content re-encode or takedown request) within 60 seconds across all PoPs
• Support at least 200 concurrent open connections per appliance per title being streamed
• Serve all bitrate variants of a title (240p through 4K HDR), totaling 10-60 GB per episode

Non-Functional Requirements

• Cache hit rate: greater than 95% for top-1000 titles at any active PoP
• Bandwidth: sustain 600 Gbps peak throughput across North America PoPs
• Latency: p99 TTFB under 50ms from PoP to user device
• Storage: each Edge PoP appliance holds 100TB (flash NVMe) for hot content; Regional Clusters hold 5PB (mixed HDD+SSD) for the warm tier
• Availability: 99.99% video start success rate (VSS); a single PoP failure must not interrupt any active streams
• Prefetch completion: nightly cache fill must complete before peak viewing hours (8pm-midnight local time)
• Invalidation: cache purge propagation under 60 seconds to all affected PoPs globally

Constraints and Assumptions

• We focus on long-form video delivery (episodes, movies); live streaming uses a separate pipeline
• All video is pre-encoded; adaptive bitrate streaming (ABR) selection happens on the client device
• Content catalog changes slowly relative to the prefetch cycle (daily additions, not real-time)
• ISP-hosted PoPs have dedicated 40Gbps-100Gbps fiber uplinks to the ISP backbone
• We assume DNS-based steering resolves the correct PoP for each user

High-Level Architecture

The system has five major components: the Steering Service (routes user requests to the optimal PoP), the Edge PoP Appliances (serve cached content to users), the Regional Clusters (origin shield tier, absorbs misses), the Cache Advisor (computes nightly fill decisions), and the S3 Origin (authoritative source of all encoded video).

When a user presses play, their Netflix app requests a manifest file from the API tier. The manifest contains segment URLs pointing at the CDN. The client’s first segment request goes to the Steering Service - a DNS-based redirector that knows the health, load, and geographic proximity of every active PoP. The Steering Service returns a redirect to the optimal PoP. If that PoP has the segment cached, it serves the bytes directly from NVMe flash. If not (a cache miss), the PoP checks its sibling appliances in the same cluster via a consistent hash lookup. If the segment is not in any appliance in the cluster, the request escalates to the Regional Cluster (origin shield), which has a broader 5PB cache. Only if the Regional Cluster also misses does a request reach S3 origin - an event that should happen fewer than 5% of the time.

The Cache Advisor runs as an offline batch job each afternoon. It ingests the previous 7 days of view logs, applies a popularity-forecasting model per PoP region, weights the predictions by content release schedule (a new season dropping tomorrow gets mandatory placement regardless of current popularity score), and outputs a ranked fill list for each active PoP. The fill list is pushed to the appliances as a download queue. Appliances pull their fill content from the Regional Cluster’s warm tier during off-peak hours (typically 2am-6am local time), avoiding contention with live user traffic.

The S3 Origin holds the master encoded files for every title in every bitrate variant. These are immutable once created; any re-encode creates a new object with a new URL. Content takedowns work by removing the file from origin and pushing an invalidation directive to all PoPs - the cached copy becomes unreachable because the Steering Service stops routing to its URL before the cached bytes are physically deleted.