System Design - data-engineering
#01 Data Engineering

Build a RAG Pipeline for an Enterprise Knowledge Base Premium

Design a Retrieval-Augmented Generation pipeline that ingests internal documents, chunks and embeds them, retrieves the most relevant context for each query, and re-ranks results before passing them to the LLM.

Read
#02 Data Engineering

Build a Real-Time Analytics Dashboard at 100B Events Premium

Design a product analytics system that ingests billions of events per day, supports sub-second aggregate queries over arbitrary time windows, and powers live dashboards without precomputing every possible metric.

Read
#03 Observability

Build a Real-Time Log Aggregation Pipeline Premium

Design a log ingestion and querying system that handles 1 million events per second, supports full-text search with sub-second latency, and retains 90 days of logs without breaking the budget.

Read
#04 Data Engineering

Build a Real-Time ML Feature Store Premium

Design a feature store that serves pre-computed features for online ML inference in under 5ms, keeps features consistent between training and serving environments, and supports point-in-time correct feature lookups for offline training.

Read
#05 Data Engineering

Build Reddit's Post Ranking Algorithm Premium

Design a post ranking system that combines upvotes, downvotes, time decay, and engagement signals into a score that keeps feeds fresh without requiring full recomputation on every page load.

Read
#06 Distributed Systems

Build a Schema Registry for Event-Driven Systems Premium

Design a schema registry that enforces backward and forward compatibility for Kafka topics, prevents breaking changes from reaching production, and lets consumers evolve independently of producers.

Read
#07 Databases

Build a Semantic Search Engine with Hybrid Ranking Premium

Design a search engine that combines keyword-based BM25 scoring with dense vector similarity to handle queries where exact keyword matches are absent but semantic intent is clear.

Read
#08 Data Engineering

Build Spotify's Song Radio Feature Premium

Design a real-time playlist generator that uses a seed song to produce an infinite radio stream of acoustically and contextually similar tracks, personalizes the mix per listener, and avoids repetition across sessions.

Read
#09 Databases

Build a Time-Series Metrics Store Premium

Design a metrics storage engine that compresses billions of data points efficiently, supports fast range queries for dashboard rendering, and retains high-resolution data for 15 days before downsampling.

Read
#10 Scalability

Build Twitter's Trending Topics Pipeline Premium

Design a system that detects emerging viral signals from a stream of 500,000 tweets per second, computes trending topics per region, and refreshes trends every 60 seconds without reprocessing the full history.

Read