Vertical vs Horizontal Scaling: Choosing How Your System Grows


Your startup just got featured on a major news site. Traffic spikes 20x in an hour. Your single application server is at 100% CPU. You have two options: call your cloud provider and upgrade to a larger instance, or spin up 19 more instances behind a load balancer. Both will work right now. But they lead to completely different architectures six months from now.

This is the vertical vs horizontal scaling decision. It is not just a technical choice - it determines how your system handles failure, how much it costs at scale, and how complex your codebase needs to be.

What they actually mean

Vertical scaling (scaling up) - Replace your current machine with a bigger one. More CPU cores, more RAM, faster storage. The application does not change. You just get more resources on a single node.

Horizontal scaling (scaling out) - Add more machines running the same application. Distribute the load across multiple nodes. Requires the application to work correctly when running as multiple instances simultaneously.

graph TB
subgraph vertical["Vertical Scaling"]
  V1["Server
4 CPU, 16GB RAM
1000 req/s"]
  V2["Bigger Server
32 CPU, 128GB RAM
8000 req/s"]
  V1 -->|"upgrade"| V2
end

subgraph horizontal["Horizontal Scaling"]
  H1["Server 1
4 CPU, 16GB RAM"]
  H2["Server 2
4 CPU, 16GB RAM"]
  H3["Server 3
4 CPU, 16GB RAM"]
  LB["Load Balancer"]
  LB --> H1
  LB --> H2
  LB --> H3
end

style V1 fill:#F1EFE8,stroke:#888780,color:#444441
style V2 fill:#EEEDFE,stroke:#534AB7,color:#3C3489
style LB fill:#E1F5EE,stroke:#0F6E56,color:#085041
style H1 fill:#EEEDFE,stroke:#534AB7,color:#3C3489
style H2 fill:#EEEDFE,stroke:#534AB7,color:#3C3489
style H3 fill:#EEEDFE,stroke:#534AB7,color:#3C3489

How vertical scaling works

You stop the instance, resize it, restart it. On AWS, this means changing the instance type (t3.medium to m5.4xlarge). On a physical server, it means adding RAM sticks or replacing the CPU.

The application sees more resources and uses them automatically - more threads can run in parallel, more data fits in memory, queries execute faster. No code changes required.

The ceiling: the largest available instance. AWS’s largest general-purpose instance (u-24tb1.metal) has 448 vCPUs and 24TB of RAM. Beyond that, you cannot scale vertically. And well before that ceiling, the cost curve becomes brutal - doubling resources often costs more than double.

How horizontal scaling works

You run multiple copies of your application behind a load balancer. Each copy handles a subset of requests. When load increases, you add more copies. When load decreases, you remove them.

This sounds simple but requires the application to be stateless - it cannot store session data, cache, or any user-specific state in local memory. If a user’s first request hits Server 1 and their second request hits Server 2, Server 2 must be able to handle it without knowing what Server 1 did.

State must live outside the application: in a database, a shared cache (Redis), or a distributed session store.

graph LR
subgraph stateful["Stateful - Cannot Scale Horizontally"]
  S1["Server 1
Session: user=alice
cart=[item1,item2]"]
  S2["Server 2
No session data
Returns error"]
  U1["Alice"] -->|"request 1"| S1
  U1 -->|"request 2"| S2
  S2 -->|"session not found"| U1
end

subgraph stateless["Stateless - Scales Horizontally"]
  SS1["Server 1"]
  SS2["Server 2"]
  RD["Redis
Session store"]
  U2["Alice"] -->|"request 1"| SS1
  SS1 -->|"read/write session"| RD
  U2 -->|"request 2"| SS2
  SS2 -->|"read/write session"| RD
end

style S2 fill:#FCEBEB,stroke:#A32D2D,color:#791F1F
style RD fill:#E1F5EE,stroke:#0F6E56,color:#085041
style SS1 fill:#EEEDFE,stroke:#534AB7,color:#3C3489
style SS2 fill:#EEEDFE,stroke:#534AB7,color:#3C3489

Where it breaks or gets interesting

Databases are the hard part

Stateless application servers scale horizontally with ease. Databases are the bottleneck. A single PostgreSQL primary can handle maybe 10,000-50,000 simple queries per second. Beyond that, you need read replicas (horizontal read scaling), sharding (horizontal write scaling), or a database designed for horizontal scale like Cassandra or CockroachDB.

This is why “just add more servers” often does not fix database-bound applications. The application layer scales, but all those new servers hammer the same database.

Vertical scaling has a hidden advantage: simplicity

A single large server avoids distributed systems problems entirely. No network calls between nodes, no consistency issues, no distributed transactions. A 96-core machine with 384GB RAM can handle enormous workloads. Many successful companies run on a handful of very large servers rather than hundreds of small ones. Stack Overflow famously serves millions of requests per day from a small number of powerful servers.

Horizontal scaling changes your failure model

With one server, failure is binary: it is up or it is down. With 10 servers, one can fail and the other 9 absorb the load. But you now have 10x more components that can fail. You need health checks, load balancer configuration, graceful shutdown handling, and distributed tracing to understand what is happening across all instances.

Auto-scaling is horizontal scaling with automation

Cloud providers let you define scaling policies: add an instance when CPU exceeds 70%, remove one when it drops below 30%. This handles traffic spikes automatically. But auto-scaling has lag - spinning up a new instance takes 1-3 minutes. For sudden spikes, you need either pre-warming or enough headroom that the existing instances can absorb the spike while new ones start.

Real-world systems and their approaches

Vertical scaling dominant:

  • PostgreSQL primary - Most teams run a single large primary. Vertical scaling buys significant headroom before sharding is needed.
  • Redis - Single-threaded for commands. A large instance with fast CPU handles millions of operations per second.
  • Stack Overflow - Runs on a small number of very powerful on-premises servers. Simplicity over distributed complexity.

Horizontal scaling dominant:

  • Stateless API servers - Every major web company runs hundreds to thousands of application server instances behind load balancers.
  • Cassandra - Designed from the ground up for horizontal scaling. Add nodes to increase both capacity and throughput linearly.
  • Kafka - Partitions distribute across brokers. Add brokers to increase throughput.
  • Kubernetes workloads - The entire model is horizontal: pods are small, stateless, and scaled by replica count.

Both, at different layers:

  • AWS RDS - Vertical scaling for the primary (instance size). Horizontal scaling for reads (read replicas). Sharding for writes beyond a single instance.

How to apply it in practice

Start vertical, move horizontal when necessary

For most applications, start with a single well-sized server. It is simpler to operate, easier to debug, and often cheaper at low scale. Move to horizontal scaling when you hit one of these limits:

  • The largest available instance is not enough
  • You need zero-downtime deploys (rolling deploys require multiple instances)
  • You need geographic distribution
  • A single instance is a single point of failure you cannot tolerate

The stateless checklist

Before horizontally scaling your application, verify:

  1. No in-process session storage (move to Redis or a database)
  2. No local file writes that other instances need to read (move to S3 or shared storage)
  3. No in-process caches that must be consistent across instances (move to Redis or accept staleness)
  4. Background jobs are idempotent (multiple instances might pick up the same job)
  5. Scheduled tasks run on only one instance (use a distributed lock or a dedicated job runner)

Cost comparison

At low scale, vertical is often cheaper - one large instance vs multiple small instances plus a load balancer. At high scale, horizontal wins because you can use commodity instances and scale precisely to demand (pay for exactly what you need, not the next tier up).

FAQ

Q: Can you do both at the same time?

Yes, and most production systems do. You vertically scale each individual instance to a reasonable size (not the smallest, not the largest), then horizontally scale the number of instances. This gives you the simplicity benefits of larger instances while still distributing load and avoiding single points of failure.

Q: What is diagonal scaling?

A term sometimes used to describe scaling both vertically and horizontally simultaneously - increasing instance size while also increasing instance count. Useful during rapid growth when you need maximum headroom quickly.

Q: Does horizontal scaling always improve availability?

Not automatically. If all your instances share a single database and that database goes down, all instances fail together. Horizontal scaling improves availability only when the failure domains are actually independent. Spreading instances across availability zones, using redundant databases, and eliminating shared single points of failure is what actually improves availability.

Interview questions

Q1: Your API server is CPU-bound at peak load. You have doubled the instance size twice already. What do you do next?

Strong answer: First, profile to understand what is consuming CPU. If it is inefficient code (N+1 queries, unnecessary computation), fix the code - that is cheaper than more hardware. If the CPU usage is legitimate work, consider horizontal scaling: make the application stateless if it is not already, put it behind a load balancer, and run multiple instances. Also consider whether some CPU-intensive work can be moved to async background jobs, reducing the load on the request-handling path. If the bottleneck is a specific operation (like image processing or PDF generation), consider offloading it to a dedicated service that can scale independently.

Q2: You are designing a system that needs to handle 10x traffic spikes during flash sales. How do you approach scaling?

Strong answer: Flash sales are a classic auto-scaling problem. The application layer should be stateless and horizontally scalable with auto-scaling policies. Pre-warm instances 15-30 minutes before the sale starts - do not rely on reactive auto-scaling alone because the spike will hit before new instances are ready. Use a queue to absorb burst traffic: accept orders into a queue immediately (fast, scalable) and process them asynchronously (database writes, inventory checks). This decouples the user-facing latency from the backend processing capacity. For the database, ensure read replicas handle catalog reads so the primary only handles writes. Consider a read-through cache for product data that does not change during the sale.

Q3: A colleague argues that horizontal scaling is always better than vertical scaling. How do you respond?

Strong answer: It depends on the workload and the operational complexity you can handle. Horizontal scaling is better for stateless services that need high availability and can tolerate distributed systems complexity. But for stateful systems like databases, horizontal scaling introduces significant complexity: distributed transactions, consistency trade-offs, resharding operations. A single large PostgreSQL instance is often the right choice up to very high scale because it avoids all of that complexity. The operational cost of running and debugging a distributed system is real. Vertical scaling also has better per-core performance for CPU-bound workloads because there is no network overhead between components. The right answer is: use vertical scaling until you hit its limits or need specific distributed properties, then move to horizontal.