Horizontal vs Vertical Scaling

Your application runs comfortably on one server until a busy launch brings more requests. You could give that server more resources, or run additional servers and distribute requests among them. These choices change capacity, failure behaviour and the work needed to operate the application.

Vertical scaling means making a machine larger, such as adding CPU or memory. Horizontal scaling means adding more machines or application instances. A system can use both. The useful choice depends on what limits the application and whether its work can be shared.

A database may need more memory to keep frequently used data available without repeated disk reads. A larger machine can help without introducing another database owner. A CPU-heavy task may benefit from a faster processor or more cores if its software can use them.

More resources are not a promise of proportional speed. A task using one core may gain little from extra cores; a slow remote dependency may remain slow after adding memory. Measure the constrained resource first.

A larger machine has a practical size and cost limit. Resizing may involve a restart or migration, depending on the platform. It also leaves one machine as a failure boundary unless you separately design redundancy and recovery.

Several application instances can receive requests through a load balancer. This helps when instances can perform work independently and the traffic can be distributed. Adding a second instance does not automatically make one request finish in half the time.

Keep required information available to any suitable instance. A shopping cart saved only in one process may disappear when a later request reaches another process. Store important state in an appropriate shared durable service, or deliberately design routing and recovery around local state.

More instances create deployment, health-check and coordination work. They may share a database, queue or network limit. Also place them across meaningful failure boundaries: several processes on one machine still share that machine’s failure.

This model gives one small application server an illustrative budget of 100 requests per second. Vertical mode uses one server with a 100, 200 or 400 RPS budget. Horizontal mode uses one to four independent 100 RPS servers. Change the incoming demand and compare the remaining capacity after one server fails.

Enable the shared database limit. Every request is assumed to need one database operation, and the database can serve only 250 of those operations per second. The useful capacity becomes the smaller of application and database capacity. Adding application capacity beyond that limit cannot complete more work.

These are teaching assumptions, not measured hardware specifications. Distribution is perfect, scaling is immediate, requests are equally expensive, and routing and dependencies stay healthy apart from the selected failure. Unserved demand is the rate beyond the model’s capacity; it could become waiting or rejection in a real system.

Autoscaling changes capacity using rules and observed metrics. A rule might add instances when demand or queue age rises, then remove them when demand falls. New instances still need to start, load their application and pass readiness checks before receiving traffic.

Choose a signal that reflects the constrained workload. CPU can help with CPU-heavy work; it can miss requests waiting on a database. Use minimum and maximum capacity, stable scaling behaviour and a plan for a sudden spike. Scheduled capacity can help when a busy period is predictable.

When removing an instance, stop sending new work and allow existing work to finish within a defined deadline. This is often called draining. Scaling down too aggressively can interrupt requests or cause repeated capacity changes. Automation does not replace testing failures, delays and operating limits.

Official sources checked 6 October 2026. These tools manage different units of capacity. None provides one universal application throughput figure: benchmark your payloads, dependencies, configuration and error/latency targets.

Amazon EC2 Auto Scaling

EC2 Auto Scaling manages groups of virtual machines with minimum, desired and maximum sizes and scaling policies. Evaluate it for applications hosted on EC2. AWS states there is no additional fee for this service itself; the underlying resources still incur charges. Account for boot time, health checks and database capacity.

Kubernetes Horizontal Pod Autoscaler

The Horizontal Pod Autoscaler changes the number of application replicas using configured metrics. It is part of open-source Kubernetes, but running a cluster and hosting replicas costs resources and operating effort. Extra replicas need available cluster capacity; increasing replica count is different from adding host machines.

Azure scaling approaches

Azure’s autoscaling guidance explains scaling considerations across workloads. Evaluate the specific hosting service’s scale-out and resize controls, supported signals and limits. Managed hosting can reduce some maintenance while introducing service-specific rules and billing.

A small application is short of memory

A larger machine may be the simplest next step if measurement confirms the memory limit. Check resize disruption, cost and recovery. You do not need a distributed application solely because horizontal scaling exists.

Many independent web requests need more application capacity

Evaluate additional instances and load balancing. Make sessions and required data available across instances, and test the loss of one instance. The lab shows why enough healthy capacity must remain after the failure.

All instances wait for the same database

Adding application instances can increase pressure without increasing useful throughput. Inspect queries, indexes, connection limits and suitable caching before choosing a database scaling strategy. Sharding is another design choice with its own data-placement and recovery work.

Compare the entire running cost, deployment effort and failure behaviour. A larger server and multiple smaller servers are not automatically equal in either price or measured performance.

A stateless request worker does not rely on its own local memory to recover the visitor’s required conversation state. The overall system still has state: carts, sessions and orders live in appropriate stores. A request can then reach another worker without losing those records.

Putting a cart only in instance A’s memory makes instance B unable to read it. Sticky routing keeps a visitor near A, but A’s failure still loses that local cart unless there is another recovery mechanism. A shared session store or appropriate signed session representation can change the contract; revocation and sensitive information still need careful handling.

Move session state deliberately, verify old and new instances can read it during rollout, and measure the shared store’s capacity. A shared store is a new dependency rather than unlimited reliability. See compute models for the same issue with invocation-local state.

Vertical scaling increases one machine’s resources. Horizontal scaling adds places to run work. Both require measurement, and both can be useful in the same system.

Check state placement, shared dependencies and remaining capacity after failure. Autoscaling needs meaningful signals, startup time and safe removal. Choose the next step that meets the workload and recovery goal with manageable cost and complexity.