Capacity Planning — Interview Q&A

Q: How is capacity planning different from setting SLOs, and why do you need both?

A: An SLO defines the reliability target you're promising — say, 99.9% of requests succeed. Capacity planning is the separate discipline of making sure enough actual infrastructure exists to hit that target under real and forecast load. You can have a perfectly reasonable SLO and still breach it constantly if capacity isn't planned to support it — the SLO tells you when you're in trouble (error budget burning), and capacity planning is what you do beforehand so that error budget isn't spent on a preventable capacity shortfall in the first place.

Q: Why is planning against average utilization a mistake?

A: A service can average a comfortable 40% CPU across a full day while still regularly spiking to 85%+ during a specific peak window — averaging smooths that spike out of visibility entirely. If headroom is sized against the average, the system is under-provisioned for exactly the moments that actually matter, since incidents happen during real peaks, not during the smoothed-out average. Capacity planning has to be based on measured peak utilization, ideally sampled over a long enough period to capture the real recurring peak pattern, not a single day that might not represent it.

Q: What's the difference between load testing, stress testing, and soak testing, and why would you run all three?

A: Load testing confirms the system holds up at expected peak traffic — validates the common case. Stress testing deliberately pushes well beyond expected peak to find the actual breaking point, which tells you your real safety margin rather than just confirming the expected case is fine. Soak testing runs sustained moderate load over hours or days specifically to catch problems a short test can't — memory leaks and slow resource degradation that simply haven't had time to manifest in a 10-minute test. Skipping soak testing is a common, real gap: teams that only run load and stress tests can ship a memory leak that a multi-hour soak test would have caught before it ever reached production.

Q: How would you size headroom for a service — walk through your actual reasoning, not just "add 30%"?

A: Headroom exists to cover the gap between a scaling trigger firing and new capacity actually serving traffic, so the real inputs are the autoscaler's measured — not theoretical — reaction time, and the fastest realistic traffic ramp rate for that specific service. A service with fast, reliable autoscaling and smooth, predictable traffic needs meaningfully less headroom than one with slow scaling and spiky traffic; using the same flat 30% for both either wastes money on the first or under-protects the second. The 20-40% range commonly cited is a reasonable starting point, but treating it as a fixed constant rather than deriving it from the specific service's actual reaction time and traffic profile is the mistake to avoid.

Q: What's a "thundering herd" in the context of autoscaling, and how do you prevent it?

A: It's a secondary failure caused by the scale-out event itself, not the original traffic spike — when many new replicas start simultaneously and all perform the same expensive startup action at once (opening a full connection pool against a database, warming a cache, fetching config from a shared service), that surge can overwhelm a dependency that was perfectly healthy under normal steady-state load. The fix is staggering startup work — jittered delays before expensive initialization steps, connection pools that ramp up gradually rather than opening every connection immediately — so a scale-out event doesn't become a self-inflicted outage on a shared dependency.

Q: Why is capacity planning for a database fundamentally different from capacity planning for a stateless API service?

A: Stateless services scale horizontally in a straightforward way — add more replicas, load-balance across them, and both read and write capacity scale together. Databases break that: read capacity can genuinely scale via replicas, but write capacity in most relational databases doesn't scale the same way — it's bounded by vertical scaling (a real ceiling on a single primary) or requires sharding, which is a significant engineering project with real lead time, not a capacity-planning knob you turn on demand. Planning has to forecast read and write load separately for exactly this reason — they have different scaling paths and completely different lead times to actually execute if more capacity is needed.

Q: How would you validate that a multi-region failover architecture actually has enough capacity, not just that it's designed to fail over?

A: A design that "should" support failover but has never been tested under real load is an untested assumption, not a verified fact — the way to actually validate it is provisioning the failover region with steady-state headroom that already accounts for absorbing a failed region's full traffic, and then testing that assumption directly, ideally via a chaos experiment that actually removes the primary region's capacity and measures whether the secondary genuinely holds under the real resulting load. Provisioning each region only for its own normal traffic, with no accounted-for failover headroom, means a real regional failure doubles load onto a region that was never sized for it.