Imagine a law firm with three offices - London, New York, and Singapore - each keeping physical copies of every contract. When a partner in London marks up a document, a courier runs to the photocopier, makes two new copies, crosses out the old paragraph numbers and writes the deltas in the margins, then hops on a plane to hand-deliver corrected pages to each office. That is roughly what naive file sync does: copy the whole file, ship it everywhere, hope nobody else edited it while the courier was airborne. The moment two offices mark up the same contract over the weekend, you have a problem.

Google Drive, Dropbox, iCloud, and OneDrive collectively synchronize data for billions of users across petabytes of storage and millions of concurrent connections. A typical Drive deployment handles hundreds of thousands of file mutations per second and must propagate each change to every connected device in under 30 seconds. An average user has roughly 10 GB stored and edits files from three to five devices. At that scale, uploading the entire file on every keystroke would consume roughly 50 Tbps of upload bandwidth globally - more than most internet providers deliver.

The naive approach fails for three reasons. First, bandwidth is not free: mobile users on metered LTE plans cannot afford multi-megabyte uploads for a one-sentence edit. Second, latency compounds across the round trip - detect change, compress, upload, fan-out, download, apply - making a 30-second freshness target impossible if each step moves gigabytes. Third, offline edits on two devices collide when they reconnect, and a first-write-wins strategy silently destroys work.

Four forces pull against each other in every design decision: bandwidth efficiency versus freshness (you can batch uploads to save bytes but you sacrifice latency), consistency versus offline availability (locking a file to prevent conflicts prevents you from working offline), simplicity versus conflict handling (the easiest conflict strategy - “last writer wins” - is the most dangerous), and storage cost versus version history (keeping every version enables recovery but multiplies storage spend). We need to solve for three core problems: detecting changes as they happen on any device, propagating only the bytes that changed rather than the whole file, and resolving conflicts that inevitably occur when the same file is edited from two devices without an internet connection.

Requirements and Constraints

Functional Requirements

• Detect file changes on any connected device within 1 second of the change occurring
• Upload only changed portions of a file (delta sync) rather than the full file on every change
• Propagate changes from the originating device to all other linked devices within 30 seconds
• Detect concurrent edits to the same file and resolve conflicts without silently losing data
• Maintain version history for at least 30 days, allowing point-in-time recovery
• Support full offline editing and queue sync operations until connectivity is restored
• Deduplicate identical file content across users to minimize storage consumption

Non-Functional Requirements

• Sync propagation under 30 seconds end-to-end for files under 1 GB on a stable connection
• 99.99% availability for the metadata service (four nines - less than 53 minutes downtime per year)
• Petabyte-scale object storage with per-file deduplication at the chunk level
• Exactly-once delivery semantics for the sync event queue
• Support 1 billion users with average 10 GB stored each (10 exabytes total raw storage)
• API throughput of at least 500,000 metadata operations per second at peak

Constraints and Assumptions

• Mobile clients must minimize battery and data usage - prefer background sync and cellular-aware scheduling
• Clients operate on unreliable connections - partial uploads must be resumable without restarting
• File sizes range from 1 KB notes to 50 GB video exports; the chunking strategy must handle both
• We assume the cloud object store (Google Cloud Storage or S3-equivalent) is reliable and handles its own replication

High-Level Architecture

The system has seven major components. The Change Detector runs on each client device, watching the file system for mutations. The Chunk Engine splits files into content-defined chunks and computes fingerprints for each. The Sync Queue is a durable event log (think Kafka) that orders sync events per user. The Metadata Service is the source of truth for file state, chunk inventory, and device clock vectors. The Object Storage layer (GCS or S3-equivalent) holds the actual chunk bytes, deduplicated across all users. The Vector Clock Service tracks concurrent edit ordering across devices. The Conflict Resolver detects divergent histories and creates conflict copies when automatic merge is not safe.

The data flow for a standard sync cycle: a file changes on Device A, the Change Detector picks it up and hands off to the Chunk Engine, which computes the diff against the last known chunk manifest. Only new or modified chunks are uploaded to Object Storage. The Chunk Engine then writes a sync event to the Sync Queue containing the file ID, new chunk manifest, and the device’s current vector clock. The Metadata Service consumes from the queue, checks the vector clock against the stored clock for that file, and either applies the update or routes it to the Conflict Resolver. The Metadata Service then pushes a notification to all other devices linked to the same user account via long-polling or WebSocket, telling them a new version is available. Each recipient device downloads only the chunks it does not already have locally.

The Change Detector

Think of the Change Detector as the building’s security guard - it sits at the door watching every person (file) who enters or leaves and logs the event with a timestamp. On desktop platforms (macOS, Linux, Windows), the operating system provides native file system notification APIs: FSEvents on macOS, inotify on Linux, ReadDirectoryChangesW on Windows. These APIs deliver kernel-level events the moment a file descriptor is closed after a write - no polling, no wasted CPU.