Volume II · Part IV · Chapter 18

Reserve Capacity

Timothy O’Brien 13 min read Free online

Show me a large amount of idle capacity and I want to understand it. I may also want to leave it alone, at least until someone can explain what happens if we take it away. That can be an uncomfortable answer in a cost review, especially when the chart appears to have done the investigation for us: here are the resources, here is the low utilization, and here is the proposed saving.

Perhaps the resources are waiting for a recovery, a customer deadline, or maintenance whose timing is difficult to predict. Perhaps the event that justified them has already passed and nobody has revisited the hold. Both explanations are possible, which is why I begin with the work the reserve protects and the conditions under which it becomes useful. Calling it insurance does not establish that it is adequate; calling it waste does not establish that we can release it.

A reserve is where Volume I, Chapter 1’s two dashboards most often meet. Release it to satisfy the cost view, and the capacity view may return the problem months later as an incident. Frédéric Bastiat described the asymmetry in 1850: an act produces a series of effects, and only the first is seen; the others unfold later, and they are not seen unless someone foresees them.1 The saving from releasing a reserve is seen on the day of the release. The incident the reserve would have absorbed stays unseen until it happens, and if the reserve is kept, it is never seen at all. That is why reserves are so easy to cut, and why the case for one has to be made before the event rather than after it.

How to read this chapter

This chapter is about capacity held for failure and growth: readiness, borrowing, ownership, and the conditions for release.

  1. Available for what? — readiness, and the allowance for failure inside apparently spare capacity.
  2. Borrowed capacity changes the failure calculation — the borrowed workers, the largest-group rule, and the fifth group.
  3. Borrowing is an agreement with an end — the failure promise from day 2 to day 70, and the cost of bridging the gap.
  4. Reserve has several owners — funding, conditional sharing, and who decides.
  5. The cost of waiting for understanding — holding a reserve while the evidence is incomplete.
  6. Decide what would permit release — the criteria that free a reserve.
  7. A reserve is a choice about a scenario — the failures it does not cover.

The lesson to carry: know what the reserve protects, how quickly it can be used, and whether a borrowing agreement lasts until replacement supply is ready. Any gap needs another agreed plan.

Available for what?

A worker that takes two days to prepare for another service can help with next week’s launch. It does nothing for a surge arriving in the next minute, or for a failure group lost tonight. Even a database replica that preserves the data may lack the tested throughput to carry the full service. I want the specialists to walk the path from the resource we have to the work we expect it to perform: placement, software, access to data, credentials, network paths, quotas, and the people or automation needed to act. Until they have, a pool called “spare capacity” mostly makes our inventory look more reassuring than our options.

Azure’s on-demand capacity reservation is one way to secure the compute portion of that path. A reservation holds a quantity of one VM size in a region or availability zone, under a capacity SLA, and a VM uses it only by naming the reservation’s group. In a region with availability zones, a group created without zones goes into a zone Azure selects;2 if that zone fails, the hold fails with it. In a recovery plan, I would put the reservation’s status and zone beside the tested time to start the service. We have paid to hold compute; the specialists still need to show that data, networking, and application startup meet the recovery deadline.

Return to the constructed portfolio’s web service, whose generic failure groups do not specify a provider’s regional or zonal layout. It has 64 qualified workers in four groups of sixteen, at a synthetic 250 requests per second per worker for the specified workload mix and quality. That gives us 16,000 in normal operation and 12,000 after losing one group, against a baseline peak of 10,000.

As Volume I, Chapter 2 showed, a report comparing only normal capacity with demand shows 6,000 requests per second available, but 4,000 of that room disappears with the failed group, leaving 2,000 under the condition we have promised to support. If we offer the whole 6,000 to a new customer, we have spent part of the allowance for failure without deciding to change that promise.

A utilization report makes the same mistake in percentages. Measured against tested capacity rather than read from a CPU chart, the baseline peak uses 10,000 ÷ 16,000 = 62.5 percent, and the promise caps peak use at 48 ÷ 64 = 75 percent. Counting the other 37.5 points as spare includes a whole group, 25 points, held for the day a group fails. That group is what the promise costs: 16 workers × $40 = $640 a day at the portfolio’s price, a quarter of the service’s 64 × $40 = $2,560. Whether the promise is worth it is a question a business owner can answer.

AWS describes the same arithmetic for some of its own stateless services: across three Availability Zones, each zone runs at only 66 percent of its load-tested level so that the other two can absorb an impaired one. The capacity stays running because launching replacements during the impairment would depend on the control plane, and on the configuration, discovery, and credential systems a new instance needs, while something else is broken.3 That is why a failover allowance runs hot, and why it looks idle to a report that cannot see the failure it is waiting for.

The resources can serve several purposes at different times; the difficulty is establishing which purposes must still be served together. A launch does not agree to avoid the day on which a group fails merely because our plan would work better that way.

Borrowed capacity changes the failure calculation

The full rollout adds 4,000 requests per second and the staged version 2,000, taking demand to 14,000 or 12,000. Volume I, Chapter 4’s plan prepares and tests eight compatible workers from deferrable batch work in two days, then spreads them two to each group. That makes four groups of eighteen, so losing a group leaves 54 workers, not the 56 that the old deduction of sixteen would suggest: 54 × 250 = 13,500 requests per second, 1,500 above the staged launch and 500 short of the full rollout. The borrowed resources support one launch choice and not the other, a distinction “we found some spare capacity” would have left for somebody to discover later.

The largest group sets the deduction, so placement matters as much as the count. Put all eight borrowed workers in one group and it holds 24; losing it leaves 48 workers and 12,000 requests per second, where we started. The same rule is why the sixteen new workers will form a fifth group: four groups of twenty survive a loss with 60 workers, five groups of sixteen with 64. With NN equal groups, the ceiling on peak use is (N−1)/N(N-1)/N of tested capacity. Chapter 19 applies the same arithmetic to whole regions.

Configuration Normal (requests/s) After losing the largest group (requests/s) Peak-use ceiling
64 workers, four groups of 16 16,000 12,000 75%
72 workers, four groups of 18 18,000 13,500 75%
72 workers, one group of 24 and three of 16 18,000 12,000 66.7%
80 workers, four groups of 20 20,000 15,000 75%
80 workers, five groups of 16 20,000 16,000 80%

The constructed portfolio assumes these rates were tested in each configuration and that a lost group’s traffic spreads evenly over the survivors; adding workers to a real system does not establish a proportional increase in throughput. The eighty-worker rows are future configurations that need new supply and the batch workers’ return; the last also needs a qualified fifth group.

Borrowing is an agreement with an end

Volume I, Chapter 4’s proposal borrows the workers for thirty days, beginning when they are ready on day two. That covers day 2 up to, but not including, day 32. New supply is estimated to be qualified and ready on day 42, leaving ten days the launch calculation has not covered.

Until day 32 the service survives a group loss at 13,500 requests per second. When the workers go back, it returns to 12,000: exactly the staged demand, with nothing left for forecast error, and 800 short if demand reaches 12,800, the peak in Volume I, Chapter 4’s first constructed outcome. That exposure lasts ten days, or thirty-eight if supply slips to day 70. Keeping the workers would close it by changing somebody else’s commitment.

The cost follows the same calendar, at $40 per worker-day. Batch keeps running on the eight workers through the two preparation days, so the attribution starts on day 2:

Loan runs to the start of Days from day 2 Worker-days Attributed to the web service Beyond the original loan
Day 32, as agreed 30 8 × 30 = 240 240 × $40 = $9,600 —
Day 42, expected supply 40 8 × 40 = 320 320 × $40 = $12,800 10 days, $3,200
Day 70, delayed supply 68 8 × 68 = 544 544 × $40 = $21,760 38 days, $12,160

None of it is new enterprise cash, because the workers already exist. The batch owner counts something else: 544 worker-days of postponed work instead of 240, a substantially different plan rather than a small adjustment. The allocation charge does not tell us what the delayed batch work costs the business.

I want the next decision scheduled while alternatives remain available: renew the agreement, qualify another source, constrain demand, or bring the unmet obligation to someone who can change it. Work back from day 32 by the lead time of the slowest alternative we still want. Reminding everybody on day 32 leaves them solving the problem on the day the solution is required.

Reserve has several owners

The person paying for reserve may be neither its operator nor the person benefiting from it. A platform can carry an allowance on behalf of many small applications, a large customer may pay for a dedicated hold because its promise cannot depend on everyone else’s timing, and an enterprise may fund recovery centrally because no single product would pay for the whole arrangement.

I would not insist on making those cases look alike. I would insist on knowing who can use the resources, whose obligation takes priority, and who has authority to release them. Otherwise two reasonable conversations can produce two customers who each believe the same reserve belongs exclusively to them under the same condition.

Sharing works when the calls on a reserve do not arrive together, which is a claim about dependence. A pool sized for the larger of two services’ failure allowances can cover independent problems, such as a bad deployment in one and a failed host in the other. If both services spread their groups across the same zones, one zone failure calls on both allowances at once, and the pool is short by the smaller. Chapter 15 reached the same conclusion about forecast errors through covariance; here the common cause is usually on the architecture diagram. A platform change that makes customers move together is a reason to revisit the arrangement.

Reusing a recovery reserve for development or test work shows how easily the answer about authority goes missing. Azure’s shared capacity reservation groups, still in preview in October 2026, suggest exactly that reuse, yet by default the owner’s administrators cannot modify the VMs other subscriptions deploy into the group.4 On the day of the disaster, the recovery reserve may be full of somebody else’s test machines. The reuse can still be worth having if the agreement says who deallocates them, how quickly, and on whose signal, and if the recovery test includes that step.

A short record should tell the customer what it can count on and tell everyone else what they may still use, without anyone reconstructing months of email. For the two reserves in the constructed portfolio:

Entry Failure allowance Borrowed batch workers
Protects The service at tested quality after losing any one group Room for the staged launch until the fifth group operates
Quantity 16 workers, 4,000 requests/s (18 and 4,500 during the loan) 8 workers, two per group
Readiness Running in every group; nothing to launch during the event From day 2, after preparation and testing
Payer The web service, $640 a day Already paid for; $320 a day of attribution moves to web
Who may use it The web service only; lending it changes the promise The web service, from day 2 to day 32
Who may release it The responsible business executive, by changing the promise The batch owner; renewal needs a new agreement
Review or end The promise, group layout, tested rate, or demand changes Fifth group operating; renewal decided before day 32

The cost of waiting for understanding

Sometimes the specialist’s honest answer is that we do not yet know enough to release the capacity safely. A table rewrite such as PostgreSQL’s VACUUM FULL needs room for a second copy until it finishes (Volume I, Chapter 12), and the measurements may have missed the last time anyone ran one. An AI retrieval index may need rebuilding while the service keeps answering customers, and nobody has measured how long that takes under load. The report can show an idle resource and still omit the operation that gives it a purpose.

The investigation can cut either way. The companion’s constructed index must be rebuilt within thirty minutes, twenty of them available for compute. At an assumed 20,000 records per worker-minute, 12 million records need 12,000,000 ÷ (20,000 × 20) = 30 qualified workers; the sixteen available would process 16 × 20,000 × 20 = 6,400,000 in the window, barely half. If those sixteen look idle on a cost report, the report has the question backward, and only a measurement of the real rate under serving load will tell us how far short the hold is.

Keeping the resource costs money, and releasing it exposes the business to a requirement nobody has bounded. Volume I, Chapter 5’s answer was a bounded hold and a funded test, which on the reserve record means an investigation owner, the measurement that would change the decision, a date, and the holding cost. If the investigation would itself be uneconomic, the authorized owner may choose the ongoing cost or a different exposure, provided the choice is explained rather than quietly dropped.

Decide what would permit release

A reserve can outlive its original purpose remarkably easily if nobody defined an ending. A launch finishes, a migration changes, a customer departs, or the recovery design improves, while the allocation looks exactly as it did before. When authorizing a hold, agree what evidence would permit its release so that the next review can ask a concrete question.

That evidence might be completion of the migration, a revised customer commitment, a measured improvement in recovery, or qualification of a shared pool that can meet the same obligation. We should also understand what release accomplishes. Can we reduce an actual charge, make the resources useful to another customer, or avoid a future purchase? Improving a utilization metric alone may have little bearing on any of those.

Not every reserve has an end date. The failure allowance is the price of a standing promise, and only someone entitled to change the promise can release it. The borrowed workers go back to batch when the sixteen new workers are delivered, qualified, and operating in the fifth group. The service then survives a group loss at 16,000 requests per second, 2,000 above the full rollout, and the allowance falls from a quarter of the fleet to a fifth.

Delivered, qualified, and operating are separate states. An order acknowledgement, a delivery estimate, or a new entry in inventory shows progress, but I would want the new group carrying its share of production traffic before the batch owner takes the workers back.

A reserve is a choice about a scenario

All of this preparation protects against a particular single-group failure. It does not cover loss of the entire region, a change that invalidates the workload model, or simultaneous failure of dependencies we had treated as independent. We need to be able to explain that boundary without either apologizing for it or letting it disappear behind a claim that the service is redundant.

The business may accept the boundary, or its customer promise may require a wider one. Finding out can lead to another recovery design, a different commitment, more reserve, or an explicit decision to accept the exposure. It will also take us beyond the service team, because the next event may remove something that its carefully calculated fallback assumes somebody else will keep available.

Notes and sources

  1. Frédéric Bastiat, “What Is Seen and What Is Not Seen” (July 1850), in Selected Essays on Political Economy, trans. Seymour Cain, ed. George B. de Huszar (Irvington-on-Hudson, NY: Foundation for Economic Education, 1995), chap. 1, Library of Economics and Liberty, checked October 4, 2026. In Cain’s translation: “There is only one difference between a bad economist and a good one: the bad economist confines himself to the visible effect; the good economist takes into account both the effect that can be seen and those effects that must be foreseen.”↩︎

  2. Microsoft Learn, “On-demand capacity reservation in Azure”, checked October 4, 2026. Vendor documentation. Supports regional or zonal scope, the capacity SLA, explicit VM association, and Azure’s zone choice for a group created without zones. Supported VM series and deployment types are limited; Spot VMs, dedicated hosts, and availability sets are excluded.↩︎

  3. Becky Weiss and Mike Furr, “Static stability using Availability Zones”, Amazon Builders’ Library, December 2019, accessed October 3, 2026. Supports the 66 percent figure for three-zone stateless fleets and the recovery path’s dependence on the control plane and on configuration, discovery, and credential systems. A vendor’s account of its own internal services, not any customer’s.↩︎

  4. Microsoft Learn, “Share a Capacity Reservation Group (Preview)”, checked October 4, 2026. Supports the suggested reuse of disaster-recovery capacity by development, test, and job-oriented workloads and the owner’s default inability to modify consumers’ VMs. Preview terms may change; one listed limitation is that scale-set VMs in a shared group cannot be reprovisioned during a zone outage.↩︎