Think of your data as a library. When you first check out a book you keep it on your desk - instantly reachable. As weeks pass and you stop reading it, it migrates to your bookshelf. Eventually it goes to the basement archive. The key insight: when someone desperately needs a basement book, a librarian fetches it for them. The visitor never needs to know it was in the basement at all.

That is exactly the contract a hot-cold tiered storage system must deliver. Applications issue reads and writes against a single logical namespace. Underneath, a policy engine continuously classifies objects by access temperature, NVMe SSDs hold the hot fraction, object storage holds the cold fraction, and a transparent retrieval path handles cache misses without a single line of application code changing.

The engineering tension here is real and non-trivial. NVMe SSDs deliver sub-millisecond random reads and cost roughly $0.10-0.30 per GB per month. S3 Standard-IA costs $0.0125 per GB per month - a 10-24x difference. A system that stores 100 TB total but serves 90% of read traffic from the hot 5% of data can cut storage costs by 80% while keeping p99 read latency under 2ms for the overwhelming majority of requests. But achieving this without leaking complexity to callers requires solving five hard problems simultaneously: access frequency tracking at scale, a tiering policy engine that makes correct promotion and demotion decisions, transparent retrieval for cold objects, migration job design that doesn’t saturate the network, and careful management of rehydration latency for cold misses.

We need to solve for correctness of tier placement, operational safety during migration, and zero application coupling - simultaneously.

Requirements and Constraints

Functional Requirements

• Applications read and write objects via a single unified API endpoint regardless of which tier the data lives on
• The system automatically migrates objects from hot (NVMe SSD) to cold (object storage) based on access patterns and configurable age policies
• Cold objects are transparently retrieved and promoted to the hot tier on read access, with the application receiving the data without error
• A tiering policy engine evaluates objects on a configurable schedule and emits promotion/demotion decisions
• Access frequency counters are maintained per object with sliding-window decay
• The hot tier enforces a capacity ceiling with LFU-based eviction when utilization exceeds a threshold
• Object integrity is verified via checksums before deletion from source tier after copy

Non-Functional Requirements

• Hot tier read latency: p50 < 0.5ms, p99 < 2ms for objects on NVMe SSD
• Cold tier rehydration latency: p50 < 200ms, p99 < 800ms for objects fetched from S3 Standard-IA
• Migration throughput: sustain 100 MB/s of background migration traffic without impacting read SLA
• Metadata store: < 5ms for tier lookup on the read path
• Policy evaluation: full object corpus scanned within a 15-minute window at 10 billion objects
• Durability: 11 nines (S3 SLA) for cold tier; hot tier survives single node failures via RAID-10 or replica
• Availability: 99.99% read availability, cold miss returns degraded response within 30 seconds
• Scale: support up to 10 billion objects totaling 100 TB on the hot tier and 10 PB in cold storage

Constraints and Assumptions

• Application code cannot change - the system must be a drop-in layer below existing storage clients
• Object sizes range from 4 KB to 5 GB; the tiering policy is object-level, not block-level
• Write path always lands on the hot tier first - there is no direct cold write path
• The system is not a filesystem; it does not support rename or hard links as atomic operations
• Cost optimization targets S3 storage costs; S3 API request costs are treated as secondary
• A single region is the primary design scope; multi-region replication is a stretch goal

High-Level Architecture

The system divides cleanly into five layers: the Storage Proxy that intercepts all I/O, the Access Tracker that maintains frequency counters, the Tiering Policy Engine that makes classification decisions, the Migration Engine that physically moves data, and the Metadata Store that records where each object lives.

Every read starts at the Storage Proxy. The proxy performs a metadata lookup to determine the object’s current tier. If the object is hot, the proxy issues a direct read against the NVMe SSD pool and returns data in under 1ms. If the object is cold, the proxy enqueues a rehydration job, optionally returns a 503 with a Retry-After header, and a background worker fetches the object from S3, writes it to the hot tier, and updates metadata. Subsequent reads for that object hit the hot tier.

Every write lands on the hot tier first. The proxy writes to NVMe SSD, records tier=hot in the Metadata Store, and increments the object’s heat_score. The Access Tracker maintains a sliding window of access events per object using a decay function; objects that go unaccessed for configurable periods see their score drop below the cold migration threshold. The Policy Engine, running on a fixed interval, scans objects whose scores have crossed tier boundaries and emits migration decisions to the Migration Scheduler. The scheduler batches these into migration jobs, rate-limits them to avoid network saturation, and hands them to Downgrade or Rehydration Workers depending on direction.

The Storage Proxy

The Storage Proxy is the only component that callers talk to - it is the system’s face and must add no more than 1-2ms of overhead on the hot path.