Horizontal Pod Autoscaling: What Most Clusters Get Wrong

A striking share of production Kubernetes clusters — by some industry estimates, the large majority — run with no Horizontal Pod Autoscaler configured on any workload. That means every service is provisioned for its peak load, permanently, whether that peak happens once a day or once a quarter. It’s one of the largest and most fixable sources of Kubernetes overspend, and it’s usually not skipped out of ignorance — it’s skipped because a first attempt at HPA went badly (thrashing, unexpected evictions, a metric that didn’t behave the way it looked like it should) and nobody went back to fix it properly.

What HPA actually does, briefly

The Horizontal Pod Autoscaler watches a target metric — CPU utilization by default, though it supports memory and custom metrics too — against a set of running pods, and adjusts the replica count to keep that metric near a target value. If average CPU across a Deployment’s pods climbs above the target, HPA adds replicas; if it drops well below, it removes them, within minReplicas and maxReplicas bounds you set.

This is a different mechanism from Vertical Pod Autoscaling, which changes how much CPU and memory each pod is allowed to use rather than how many pods exist. The two are complementary, not competing — HPA handles variable load by changing replica count, VPA handles right-sizing the resource spec of each replica — but conflating them, or running both against the same metric without coordination, is one of the more common configuration mistakes.

The mistakes that make HPA look broken

Scaling on a metric that doesn’t actually track load. CPU is the default target, but for I/O-bound or memory-bound services, CPU utilization barely moves under load, so HPA never triggers when it should. If your service’s bottleneck isn’t CPU, scaling on CPU will look like HPA “doesn’t work,” when actually it’s just watching the wrong signal — custom metrics (request rate, queue depth) usually need to replace or supplement it.

No stabilization window, causing scale thrashing. Without a cooldown period, HPA can add replicas in response to a brief spike, then remove them seconds later when it passes, then add them again — burning through scheduling overhead and occasionally causing new pods to be killed before they’ve even finished starting up. Setting a stabilization window on scale-down specifically (a few minutes is a reasonable starting point) prevents this without meaningfully slowing down legitimate scale-up.

minReplicas set too low for actual availability requirements. HPA respects your PodDisruptionBudget, but if minReplicas is 1, there’s no redundancy to protect during a node drain regardless of what your PDB says — HPA and PDB need to be set with each other in mind, not independently. See our PodDisruptionBudget guide for how the two interact.

No maxReplicas ceiling, or one set far above real capacity. Without a sane ceiling, a runaway scaling event (a bug causing artificially inflated load, a metric misconfiguration) can attempt to scale far beyond what the cluster’s node pool can actually accommodate, which doesn’t crash anything but does generate a lot of Pending pods and confusing alerts.

Why “we tried it once and it caused problems” isn’t a reason to stay manual forever

The failure modes above are all fixable, specific, and well understood — they’re not evidence that autoscaling doesn’t work for your workload, just that the first configuration attempt missed a detail. Given that manually-sized clusters are provisioned for peak load permanently, the cost of staying manual compounds every month, while the cost of fixing an HPA configuration is a one-time, bounded piece of work.

A reasonable path back to autoscaling after a bad first experience: start with a wide stabilization window and conservative min/max bounds in a non-critical service, watch it for a week, and tighten from there once you trust the behavior. This is slower than configuring it once and moving on, but it rebuilds confidence incrementally instead of asking the team to trust a mechanism that burned them before.

Where this connects to cost, concretely

Autoscaling is the mechanism; the payoff is not provisioning for permanent peak. Combined with right-sized resource requests at the individual pod level, HPA is one of the two biggest levers most teams have available for reducing Kubernetes spend without touching architecture. Cost visibility that’s mapped to actual utilization makes the case concrete — seeing a namespace running at a fixed replica count with utilization that swings between 15% and 90% throughout the day is usually enough to make the argument for autoscaling on its own, without needing a broader FinOps push.

Frequently asked questions

What’s a reasonable target CPU utilization to start with? Somewhere around 60-70% is a common, conservative starting point — high enough to avoid over-provisioning, low enough to leave headroom for the time it takes new pods to start and become ready during a scale-up event.

Does HPA work well with spiky, unpredictable traffic? Reasonably, though very sudden spikes can outpace how quickly new pods can start and pass readiness checks — for genuinely spiky workloads, a higher minReplicas floor to absorb the first moments of a spike, combined with HPA for the sustained portion, works better than relying on HPA alone.

Should CPU-based HPA be the default for every service? No — it’s the right default for CPU-bound services, but I/O-bound or queue-processing workloads usually need a custom metric that actually reflects their bottleneck. Using CPU by default across all services regardless of their actual bottleneck is itself one of the common misconfigurations.

How does HPA interact with cluster autoscaling (adding/removing nodes)? They operate at different layers and work together: HPA changes pod replica count, and if there isn’t enough node capacity to schedule the new replicas, cluster autoscaling adds nodes to accommodate them. Both need reasonable bounds, or a runaway HPA event can trigger runaway node provisioning too.

See utilization patterns that make the autoscaling case obvious — start free with one cluster.