Imagine you’re running an apartment building. Every tenant pays rent, shares the same plumbing and electrical grid, yet expects to live as if they have the place to themselves. They can’t hear their neighbors’ conversations, their water pressure shouldn’t drop when someone on floor 3 showers, and evicting one tenant shouldn’t require shutting down the building. That’s the exact contract a multi-tenant database system must uphold - at 10,000 tenants and thousands of concurrent queries.

The naive approach is a single shared database with a tenant_id column on every table. It’s simple to build, cheap to operate, and will absolutely collapse the moment a large tenant runs a report query at 2 PM on a Tuesday. Their full-table scan eats all your connection pool slots, their ORDER BY thrashes the shared buffer pool, and suddenly every other tenant’s P99 latency spikes from 5ms to 5 seconds. You’ve let one noisy neighbor ruin the building.

The opposite extreme - a dedicated database per tenant - solves isolation perfectly but introduces operational nightmares: 10,000 database clusters to upgrade, monitor, back up, and provision. When a startup customer joins, you’re waiting 45 seconds for a database to spin up. When a tenant churns, you’re cleaning up an entire instance. And your infrastructure costs scale linearly with tenant count, not with actual usage.

We need to solve for data isolation, query interference prevention, and independent tenant operations simultaneously, all on shared infrastructure. The right answer sits between the two extremes, and getting there requires understanding the tradeoffs at each isolation layer.

Requirements and Constraints

Functional Requirements

• Store and query data for 10,000+ tenants on shared infrastructure
• Guarantee that one tenant cannot read or write another tenant’s data
• Allow individual tenant backup, restore, and migration without downtime for other tenants
• Support per-tenant query throttling and resource limits
• Enable schema migrations that can be applied per-tenant or globally
• Prevent cross-tenant query leakage in ORM layers and raw SQL

Non-Functional Requirements

• Query latency: P99 under 50ms for OLTP workloads
• Throughput: 50,000 queries per second across all tenants combined
• Noisy neighbor protection: one tenant using 100x normal query load must not affect others
• Recovery point objective: 5 minutes per tenant (independent point-in-time recovery)
• Tenant migration: complete within 4 hours for a 100GB tenant dataset
• Availability: 99.95% uptime across the fleet

Constraints and Assumptions

• Tenants are SaaS customers with similar schema (B2B SaaS model)
• Tenant sizes vary: 80% small (under 1GB), 15% medium (1-50GB), 5% large (50GB+)
• No cross-tenant joins required
• PostgreSQL as the primary database engine
• Kubernetes-based deployment for the application tier

High-Level Architecture

The architecture has four major layers: the tenant router that maps requests to the correct isolation boundary, the connection pool manager that enforces per-tenant connection limits, the storage tier with its isolation model, and the control plane that handles migrations, backups, and throttling policies.

Data flows through the system like this: an application request arrives tagged with a tenant identifier (typically from a JWT or session). The tenant router validates the tenant exists, resolves which isolation shard or schema owns that tenant’s data, and routes the connection request to the appropriate pool. The connection pool manager enforces that this tenant cannot take more than its allocated connections. Once connected, row-level security policies enforce data isolation at the database engine level - even if application code has a bug, the engine won’t return another tenant’s rows.

The control plane operates asynchronously: it watches for migration jobs, schedules per-tenant backups, applies throttle policies as tenant behavior changes, and handles the migration of individual tenants between shards when capacity needs rebalancing.

Tenant Isolation Models

The tenant isolation model is the most consequential architectural decision in this system. Get it wrong and you’re either rebuilding the database layout or suffering constant noisy-neighbor incidents.

There are three canonical models: shared table (one table, tenant_id column on every row), schema-per-tenant (one PostgreSQL schema per tenant, shared database cluster), and database-per-tenant (one dedicated database cluster per tenant). Each sits at a different point on the cost-isolation tradeoff curve.