Picture a bouncer working a rope line outside a busy nightclub who never once asks anyone for ID. He does not need to. He watches how each person approaches: pace, hesitation, whether they glance back at a car idling at the curb, whether they drift toward the door in the loose, correcting way people actually walk while scanning a crowd for a friend, or whether they walk in a mechanically straight line from the parking garage door like they were dropped in on rails. By the time someone reaches the rope, he already knows whether to wave them through, watch them a little closer once they are inside, or stop them cold. He never interrupted anyone’s night to check a card. A bot detection system built on behavioral signals is trying to be that bouncer at web scale: telling a browsing human apart from a scripted client by how a session behaves, not by demanding a credential nobody legitimate wants to produce.

Consider a mid-size e-commerce and ticketing platform serving around 800 million page views a month, spiking to 50,000 requests a second during a flash sale or a ticket drop. Independent bot-traffic studies routinely find that somewhere between 30% and 40% of that kind of traffic is not a person at all: scalper bots snapping up concert tickets before a human can click “buy”, credential-stuffing scripts testing stolen password lists against the login form, and scraper fleets hammering the product catalog for a competitor’s pricing feed. Every one of those has to be told apart from the other 60-70% of traffic that is a real customer trying to check out before the sale ends.

The naive approach is to put a CAPTCHA in front of anything that looks suspicious. That fails for two reasons that compound each other. First, CAPTCHAs cost real money: every checkbox a genuine buyer has to solve during a flash sale is a buyer who might abandon the cart, and conversion-sensitive businesses measure that cost in dollars, not inconvenience. Second, CAPTCHAs are a solved problem for anyone motivated enough to solve them: human CAPTCHA-solving farms clear a few cents per solve and modern computer-vision models handle the rest, so a challenge that annoys your real customers barely slows a well-funded scalper operation down.

A second failure mode is scoring on a single signal. Block by IP address alone and you punish an entire corporate office or mobile carrier sharing one NAT’d address while missing bots that rotate through a fresh residential proxy on every request. Rate-limit by request count alone and a patient bot that stays just under the threshold sails through, while a legitimate flash-sale crowd refreshing a product page gets throttled. Detection that actually works has to fuse many weak, individually gameable signals - how someone moves a mouse, how fast requests arrive, what an IP address has done at other sites, what a device’s hardware and software configuration looks like - into a single score, and it has to compute that score inline, on the request path, in the time it takes to render a page, without ever showing a real human a challenge they would resent. We need to solve for three things at once: collecting enough behavioral and network signal to make a confident call without adding perceptible latency, gating access on a continuous score rather than a binary allow-or-deny so genuine users never see a challenge, and building a feedback loop that keeps the model current as bot operators study our defenses and adjust their scripts to look more human, because a model trained once and left alone degrades the day a bot author notices it.

Requirements and Constraints

Functional Requirements

• Collect behavioral telemetry (mouse movement, scroll velocity, touch events, keystroke timing) client-side without adding perceptible page-load latency
• Compute a device fingerprint that fuses canvas, WebGL, font, and hardware-configuration signals into a stable per-device identifier that survives cookie clearing
• Compute request velocity signals per IP, per session, and per device fingerprint over sliding time windows
• Maintain an IP reputation store combining ASN and datacenter/proxy/Tor classification with locally observed abuse history
• Score every request or session in real time and route it to an allow, monitor, challenge, or block decision using score-based gating, with no visible challenge shown to low-risk traffic
• Provide a feedback loop that ingests confirmed bot and human labels (honeypots, challenge outcomes, chargebacks, manual review) and retrains the scoring model on a schedule
• Expose a query API so other services can fetch a session’s current score without duplicating the scoring pipeline
• Support emergency policy overrides that tighten or loosen thresholds instantly during an active attack, without a full redeploy

Non-Functional Requirements

• Decision latency: p99 under 15ms for the inline scoring call, since it sits directly on the request path
• Throughput: sustain 50,000 requests/second at peak, 18,000/second sustained average, each scored individually
• Scale: 800 million page views/month, roughly 12 million daily active sessions, around 150 million behavioral telemetry events ingested per day
• Accuracy: catch at least 95% of known automated traffic patterns while keeping the false-positive challenge rate under 0.05% of genuine sessions
• Availability: 99.99% for the decision path; the system must fail open (allow traffic) rather than fail closed if the scoring engine degrades, since blocking all traffic during an infrastructure incident is worse than briefly under-detecting bots
• Model freshness: incremental retrain daily, full retrain weekly, and an out-of-band emergency retrain within 4 hours of detecting a new evasion pattern
• Retention: 90 days of raw behavioral telemetry for retraining; aggregated reputation and score history retained longer

Constraints and Assumptions

• No CAPTCHA-first design. Visible challenges are a last resort for the highest-risk score band only, never the primary detection mechanism
• We do not design the challenge UI itself in depth (a standard JS proof-of-work or interactive check is assumed) - the focus is the detection and scoring pipeline that decides when to invoke it
• TLS everywhere and a same-origin client SDK are assumed
• We do not attempt to enumerate every individual anti-evasion heuristic in the bot-vs-detector arms race; the goal is an architecture that lets the model keep adapting, not a fixed list of today’s tricks
• A single global deployment is described in depth, with multi-region replication discussed only where reputation and feature data synchronization matters

High-Level Architecture

The system has six major components: the Signal Collection SDK, the Edge Ingestion and Velocity Service, the Reputation Store, the Scoring Engine, the Decision Gateway, and the Adversarial Feedback Loop.

The Signal Collection SDK runs in the browser or mobile app and captures pointer movement, touch events, keystroke timing, and a device fingerprint, batching them for delivery without blocking page rendering. The Edge Ingestion and Velocity Service receives those batches at the network edge, computes request-rate signals keyed on IP, fingerprint, and session, and forwards everything toward scoring. The Reputation Store is a fast, sharded key-value layer holding IP intelligence (datacenter and proxy classification, ASN, locally observed abuse history) that gets looked up on nearly every request. The Scoring Engine joins behavioral, velocity, reputation, and device features into a single feature vector and runs model inference to produce a continuous bot score. The Decision Gateway maps that score to a policy tier and executes it (serve normally, log more closely, invoke an invisible step-up check, or block), and the Adversarial Feedback Loop collects ground-truth labels from that outcome and periodically retrains the model that feeds the Scoring Engine.

A request flows like this: as a session interacts with a page, the SDK batches telemetry and flushes it every few hundred milliseconds. The edge layer increments velocity counters and looks up reputation for the request’s IP, then hands a feature vector to the Scoring Engine, which returns a score in low single-digit milliseconds. The Decision Gateway acts on that score inline, before the response is served, so a legitimate user never notices any of this happened. Separately, decision outcomes, honeypot triggers, and confirmed-fraud events stream into a label store that a scheduled retraining job consumes, producing a new model version that is shadow-evaluated against a held-out labeled set before it ever replaces the production model.

Component Deep Dives

The Signal Collection SDK and Behavioral Fingerprinting

This component’s job is to capture how a session behaves, not just what it claims to be, and to do it without the user noticing any of the collection is happening.

A bot’s mouse path looks like it was drawn with a ruler: dead-straight lines between two points at close to constant velocity. A human path wanders, overshoots the target, and corrects, and it speeds up and slows down non-linearly on the way there, a pattern HCI research calls Fitts’s law. Behavioral fingerprinting is the practice of turning that difference into a numeric feature rather than eyeballing it: sampling pointer position over time and computing statistics like path straightness and velocity variance that separate the two populations even when no single sample looks obviously wrong.

// Client-side telemetry collector - samples pointer movement and batches it
// Demonstrates: throttled event capture, canvas/WebGL fingerprint hash, beacon flush
const MAX_BUFFER = 400;
const FLUSH_INTERVAL_MS = 500;

class SignalCollector {
  constructor(endpoint, sessionId) {
    this.endpoint = endpoint;
    this.sessionId = sessionId;
    this.buffer = [];
    this.lastSample = 0;
    window.addEventListener('pointermove', (e) => this.onPointerMove(e), { passive: true });
    window.addEventListener('touchstart', (e) => this.onTouch(e), { passive: true });
    setInterval(() => this.flush(), FLUSH_INTERVAL_MS);
  }

  onPointerMove(e) {
    const now = performance.now();
    // Sample at roughly 60 events/sec max - enough resolution for curvature
    // analysis without flooding the buffer on high-refresh-rate displays
    if (now - this.lastSample < 16) return;
    this.lastSample = now;
    this.buffer.push({ t: Math.round(now), x: e.clientX, y: e.clientY, type: 'move' });
    if (this.buffer.length >= MAX_BUFFER) this.flush();
  }

  onTouch(e) {
    const touch = e.touches[0];
    if (!touch) return;
    this.buffer.push({ t: Math.round(performance.now()), x: touch.clientX, y: touch.clientY, type: 'touch' });
  }

  deviceFingerprint() {
    const canvas = document.createElement('canvas');
    const ctx = canvas.getContext('2d');
    ctx.textBaseline = 'top';
    ctx.font = '14px Arial';
    ctx.fillText('scalablesystem-fp', 2, 2);
    const canvasHash = canvas.toDataURL();
    const gl = document.createElement('canvas').getContext('webgl');
    const renderer = gl ? gl.getParameter(gl.RENDERER) : 'no-webgl';
    return {
      canvasHash,
      renderer,
      hwConcurrency: navigator.hardwareConcurrency,
      tz: Intl.DateTimeFormat().resolvedOptions().timeZone,
    };
  }

  flush() {
    if (this.buffer.length === 0) return;
    const payload = JSON.stringify({ sessionId: this.sessionId, events: this.buffer, fp: this.deviceFingerprint() });
    this.buffer = [];
    // sendBeacon survives page navigation/unload, unlike a regular fetch call
    navigator.sendBeacon(this.endpoint, payload);
  }
}

Device fingerprinting takes several individually weak entropy sources (the exact pixels a canvas renders, the reported GPU renderer string, timezone, hardware concurrency) and hashes them into a single identifier that is stable across sessions even after cookies are cleared, unlike a session cookie or a localStorage value a bot can trivially wipe. The server side of mouse movement analysis turns the raw batch into features a model can consume: