Tuesday morning, 9:47 AM. The engineering Slack is quiet - that pre-standup calm before the day kicks off. Then the first message: “site feels slow.” Within four minutes, three more identical reports. The on-call engineer opens Datadog. P99 latency on the orders API: 4.2 seconds. Normal baseline: 45ms. Connection pool utilization: 98 of 100 slots occupied. Database CPU: pegged at 98%.

No deploys in the last six hours. No infrastructure changes. No traffic spike visible in the load balancer logs. The engineer starts the checklist: check recent queries, check locks, check slow query log. The slow query log has one entry, and it has been running for 27 minutes.

SELECT
  region,
  product_category,
  SUM(revenue) AS total_revenue,
  COUNT(DISTINCT user_id) AS unique_buyers
FROM orders
GROUP BY region, product_category;

Then a DM arrives from the data team: “Hey, quick heads up - I’m running some revenue analysis on the read replica right now, is that okay?” The orders table has 2.1 billion rows. The query is doing a full sequential scan, spilling intermediate sort buffers to disk, consuming roughly 40 GB of I/O bandwidth, and has evicted most of the working set from the shared buffer pool - the very buffer pool that the application’s point-lookup queries depend on for their 45ms response times.

It is not okay.

This is the OLTP/OLAP collision problem - two fundamentally different workloads competing for the same hardware, and neither winning.

Why This Happens

OLTP (Online Transaction Processing) and OLAP (Online Analytical Processing) are opposites in nearly every dimension that matters for database performance.

An OLTP query looks like SELECT * FROM orders WHERE id = 9847263. It touches one row, uses an index, runs in 2ms, and releases its locks immediately. Your application issues thousands of these per second. The database handles them fine because each one is tiny.

An OLAP query looks like the one above. It touches every row in a 2-billion-row table, ignores all indexes (they would not help a full-table aggregate anyway), builds enormous hash tables in memory, spills to disk when those tables exceed work_mem, holds shared locks for the full duration, and competes for every I/O operation the disk can produce.

The failure chain when both workloads share a host:

Analyst runs GROUP BY on 2B-row orders table
  → Full sequential scan saturates disk I/O bandwidth
    → Shared buffer pool thrashed - OLTP working set evicted
      → OLTP point lookups now require disk reads (10ms instead of 0.1ms)
        → Query latency multiplies by 100x
          → Application connection pool fills up (queries take longer, hold slots)
            → New requests queue behind full pool
              → HTTP timeouts cascade to users
                → On-call engineer paged at 9:47 AM

The read replica does not save you here. The read replica is the same PostgreSQL engine, the same buffer pool architecture, the same I/O constraints. You moved the problem off the primary - good - but you did not eliminate it.

The Naive Solution (and where it breaks)

The instinct every team reaches for first: “analysts should use the read replica, not the primary.” This is a correct step, but it is not a solution. It is a mitigation that works until the replica is the bottleneck.