Imagine a fire station that receives one call and must immediately dispatch ambulances to every house in a city of 100 million residents - simultaneously, without clogging every road, and with the most urgent addresses reached first. That is the engineering problem when a celebrity with 100 million followers gets mentioned in a tweet. One write event from a single API call must propagate into potentially 100 million notification rows, 100 million cache entries, and 100 million push deliveries - all triggered in a matter of seconds.

The naive solution is obvious and catastrophically wrong: loop over the follower list and write a notification row to the database for every follower. At 100 million followers and 2 milliseconds per insert, that loop takes 55 hours. Even parallelized to 1,000 workers each doing 100,000 inserts, you are generating 100 million simultaneous database writes from a single event. Every one of those workers is holding a connection, competing for locks, and saturating your I/O throughput. This is the celebrity problem - a single logical event producing write amplification that is proportional to follower count.

The deeper tension is that the celebrity problem is not just about raw write volume. It is about the conflict between three forces that cannot all win at the same time. First, followers with 100 million people expect the notification to appear quickly - ideally within a few seconds, not hours. Second, the system must not sacrifice write throughput for all other users just because one celebrity posted something. Third, you cannot afford to store 100 million identical notification records for an event that 95% of those followers will never look at because they were asleep, inactive, or simply not interested. Write amplification math alone tells the story: if a celebrity posts once per hour and has 100 million followers, a naive push-to-all model generates 100 million writes per post - almost 28,000 writes per second just for one account.

Real systems like Twitter and Instagram handle this with a hybrid fanout approach: push eagerly to the small set of users who actually check their notifications frequently, and pull lazily for everyone else. The key insight is that follower counts are enormous but active user counts are manageable. We need to solve for write amplification, delivery latency SLA for active users, and graceful backpressure handling under celebrity post bursts - simultaneously.

Requirements and Constraints

Functional Requirements

• When a celebrity is mentioned in a tweet, all followers who have notification preferences for mentions receive a notification
• Notifications appear in the notification tab and as push alerts (APNs/FCM/Web Push)
• Users can configure per-account notification preferences (follow all mentions, only mentions to me, etc.)
• The system must track per-user read state so notification counts are accurate
• Notifications must be durable - an infrastructure blip should not silently drop a notification

Non-Functional Requirements

• Celebrity accounts defined as accounts with more than 1 million followers
• Maximum follower count to handle: 200 million followers
• Notification delivery latency for VIP users (last active within 24 hours): p95 under 5 seconds
• Notification delivery latency for all other followers: p95 under 30 minutes
• System must handle up to 500 celebrity mentions per second during peak events (award shows, elections)
• Write throughput: up to 50 billion notification rows per day across all celebrities
• Read throughput: 500,000 notification tab opens per second
• Availability: 99.9% for push delivery, 99.99% for notification store reads
• Storage: notification rows retained for 90 days before TTL-based deletion

Constraints and Assumptions

• Follower lists for celebrities are pre-segmented and stored in a sharded index service, not fetched live during fanout
• The is_vip_user flag is precomputed in a user tier service and cached in Redis
• We are designing the notification fanout path only - the tweet creation path and social graph are separate services
• Users who have not logged in within 30 days are excluded from fanout (their pull flag still gets set, but no write)
• Notification preferences are cached at the fanout layer with a 60-second TTL

High-Level Architecture

The system has five major layers: the ingestion API, the celebrity router, the hybrid fanout coordinator, async notification workers, and the storage layer.

The Mention API Gateway accepts incoming mention events from the tweet creation service. Each event carries the mentioned user’s ID, the mentioner’s ID, the tweet ID, and a timestamp. The gateway validates the event, deduplicates it using a short-window bloom filter to catch retry storms, and publishes it onto a Kafka topic partitioned by mentioned_user_id.

The Celebrity Router consumes from that Kafka topic and performs a follower-count lookup against the User Graph service. If the mentioned user has fewer than 1 million followers, the event is forwarded to a standard notification pipeline (a simple write queue). If they have more than 1 million followers, it is routed to the hybrid fanout coordinator. This single branching decision is the foundation of the entire design.

The Hybrid Fanout Coordinator is the heart of the system. It reads the celebrity’s follower shard index and makes two decisions simultaneously: which users get immediate push notifications (the VIP set - roughly the top 5% of active followers), and what Redis flag to set for the remaining 95% who will receive their notification lazily when they next open the notification tab. It then enqueues the VIP user IDs into a high-priority queue and enqueues shard task descriptors (not full user ID lists) into the bulk fanout queue.

The Async Notification Workers run in three pools: VIP workers that consume from the priority queue and write directly to the notification store plus trigger push delivery; shard workers that process bulk tasks in parallel, each handling one 100,000-follower shard; and a pull-model handler that watches the notification tab open events and resolves pending pull flags into actual notification rows on demand.

The Storage Layer consists of a Cassandra cluster for durable notification rows (partitioned by user ID), a Redis cluster for per-user feed caches and pull flags, a follower index service for paginating over celebrity follower lists, and the push delivery service (wrapping APNs/FCM/WebPush).

The diagram below shows the end-to-end data flow from a single mention event to a notification appearing in a follower’s tab.