AI System Design - databases
#01 Data Engineering
Build a Real-Time RAG Pipeline with Streaming Document Ingestion Premium
A blueprint for a streaming RAG ingestion pipeline that consumes 4,000 document change events per second from CDC and webhook sources, embeds only the delta, and keeps a 40 million chunk index queryable within an 8 second p99 staleness budget.
Read
#02 Databases
Build a Vector Index with HNSW and Quantization at 1B Vectors Premium
Design a billion-vector similarity index that serves 20,000 queries per second at 18ms p99 with recall@10 of 0.94, compressing 1.5TB of 768-dimensional embeddings into 352 bytes per vector across 64 shards.
Read