Imagine a postal sorting office that processes 3.5 million letters per second, and every single one needs a fraud check before it reaches its recipient - in under 100 milliseconds. That is the operating reality for Gmail’s spam detection system. At roughly 300 billion emails per day, this is one of the largest real-time machine learning inference systems on the planet.

The challenge is not just volume. It is the adversarial nature of the problem. A static rule-based system would be defeated within hours. Spammers A/B test their campaigns, rotate IP ranges, generate new domains, and adjust word patterns specifically to evade detection. Your system must learn continuously from user feedback - when someone clicks “report spam,” that signal must feed back into the classifier, improve it, and be deployed while emails continue flowing at full throughput. All of this must happen without disrupting delivery for the 99.9% of legitimate email.

Latency and accuracy are in direct tension. The most accurate classifiers - large transformer models - take hundreds of milliseconds per inference. But the pipeline must complete in under 100ms to stay within the SMTP delivery window. A naive approach of running a single deep neural network on every email would either miss the latency budget or require a GPU cluster so large it makes the cost structure untenable.

The system must also handle an extreme class imbalance. Roughly 85% of all email sent on the internet is spam, but Gmail users experience a near-spam-free inbox. The classifier precision must be very high - blocking a legitimate email costs real user trust. We need to solve for: sub-100ms classification latency, continuous model updates that don’t interrupt service, and false-positive rates below 0.05% even as spam patterns shift.

Requirements and Constraints

Functional Requirements

• Classify every inbound email as spam, ham, or uncertain before delivery
• Support multi-class classification (spam, phishing, malware, bulk, ham)
• Collect user feedback signals (report spam, not spam, moved from spam folder)
• Retrain models continuously using feedback signals
• Support A/B testing of new model versions against live traffic via shadow mode
• Propagate new model versions to all inference nodes within 10 minutes
• Provide per-sender reputation scoring updated in near-real-time

Non-Functional Requirements

• Throughput: 300B emails/day = 3.5M emails/second at peak
• Latency: p99 classification under 100ms per email
• False positive rate: under 0.05% (legitimate email classified as spam)
• False negative rate: under 1% (spam reaching inbox)
• Model freshness: new model deployed within 10 minutes of approval
• Availability: 99.99% uptime (spam detection must not block delivery)
• Storage: 90-day retention of features and decisions for audit

Constraints

• Classification must be stateless per email - no cross-email dependencies in the hot path
• SMTP protocol imposes a hard latency ceiling on the classification step
• Models must be versioned and rollback-capable in under 5 minutes
• User-reported spam must never cause false positives for other users without model retraining

High-Level Architecture

The system divides into three planes: the hot path (per-email classification, must be fast), the warm path (feedback collection and model evaluation), and the cold path (model retraining and registry).

The Email Ingest MTA receives all incoming SMTP connections and places emails onto a distributed ingest queue. Emails are deduplicated at this stage to handle retransmission storms from misconfigured senders. The Feature Extractor runs in parallel on sharded workers, pulling IP reputation data, parsing email headers (SPF, DKIM, DMARC authentication), extracting URLs for reputation lookup, and generating content n-grams - all in memory with locally cached lookup tables.

The Naive Bayes Scorer handles 80% of emails in under 5ms with a clear decision. The Neural Classifier activates only for uncertain cases (score between 0.1 and 0.7), adding 40-80ms to those emails. The Decision Router applies the final threshold and routes to inbox, spam folder, or block. Downstream, the Feedback Collection Pipeline captures user actions asynchronously and feeds the Model Trainer, which runs incremental updates and pushes new model versions through the Model Registry to all inference nodes.

Feature Extraction Service

The feature extractor’s job is to transform a raw email into a vector of ~500 numerical signals within 10ms.

Think of it like a TSA security checkpoint with multiple parallel scanners running simultaneously. Each scanner checks a different property - one checks the ID (IP reputation), one runs the bag through X-ray (content analysis), one checks the boarding pass against the list (domain authentication). All run in parallel, and the combined result takes no longer than the slowest scanner.