Introduction

This is Volume I of Scale Without Surprise, Understanding Cost, Capacity, and Capability. It examines the work, its economic consequences, and the evidence needed for a useful decision. Volume II, Managing Demand, Risk, and Scale, follows those decisions through changing demand, customer portfolios, reserves, commitments, and review. The two volumes share the constructed portfolio below; chapters retain their series numbers.

One of the first managers I worked for who understood this job offered it to me with a warning:

One of the worst jobs in IT, it’s the equivalent of passing the Kobayashi Maru test every week.

If you are trying to recruit someone, there are probably better ways to describe the opportunity. Having done the job, I will confirm the description. It is a job that often has to communicate limits to people who don’t like to think about limits. But I appreciate that he understood what he was asking me to do, because a lot of the advice you will read about capacity management assumes that the problem can be solved once everyone supplies the right inputs. Get the requirements, measure the workload, forecast demand, acquire the capacity. There you go. Why is everyone having such a difficult time?

For readers who did not grow up with Star Trek, the Kobayashi Maru is a command exercise designed as a no-win scenario. A ship is in distress, and going to its assistance exposes the rescuing crew to overwhelming opposition. Refusing to help means abandoning the people who need rescuing. The exercise is intended to reveal how a commander responds when there is no satisfactory outcome. James T. Kirk does not solve the exercise as given: he reprograms the simulation so that a rescue becomes possible. He changes the conditions of the test.¹

That is a useful way to think about this job. You still cannot reprogram a supply shortage. A difficult week can bring an ambitious launch from the business, new work from engineering, a firm reliability commitment from site reliability engineering (SRE), and a smaller budget from Finance to the same table. Each request can be reasonable on its own. Together, they may exceed the supply and time we have. Sounds easy, right?

My manager understood that the job was going to involve changing some of those conditions: finding out what a customer actually needed, discovering that an apparently fixed deadline had some flexibility, or showing that buying capacity now is cheaper than running out of it. You can do the calculations correctly and still be a long way from an answer. Some of the inputs cannot be settled until an architecture is final, and you may have to commit to a limited amount of capacity before the teams who will use it are ready to make commitments of their own. It is often the capacity manager who walks teams through those compromises.

This is the phrase I keep coming back to:

Cloud Capacity and Cost is less something to be managed as it is a challenge that will manage you if you approach it from the wrong angle.

By the wrong angle, I mean treating an early capacity estimate as settled after the conditions behind it have changed. Bringing the capacity discussion forward is useful; keeping it current is a continuing responsibility. While one component is being built, another business may change its plans, a customer may prepare a migration, and an experiment may become an executive priority overnight. The early decision gives us a starting position to revise, not an exemption from those changes.

Generative AI has made all of this more urgent and more expensive. For the ten years before it arrived, AI work had its own special license to do whatever it wanted, because it wasn’t a large part of the footprint. Somebody needs 10 A100s to do something for image search? Go ahead. Four years later, somebody needs billions of dollars for a series of racks of the rarest sort of computer on the face of the earth, not yet being manufactured, and the funds have to be committed now, with the plan to follow. A lot of companies are in that situation. What we did not build before generative AI was real discipline and accuracy in assessing AI workloads: what they would consume, what return they would earn, and how to make sure we weren’t about to misspend the money. If I were starting again in 2016, I would begin building analytical muscle around how AI works then, while it was steadily more expensive but not yet the most significant part of the job.

The pace makes it harder. A feature that calls a model can be built in days, and changing its prompt, its model, or the number of steps an agent takes can multiply what it consumes without adding a single user. The dedicated model capacity or GPUs behind it can take much longer to secure. This job is understanding the choices and the tradeoffs, but it is also understanding how to manage a Kobayashi Maru.

Cost and capacity in the context of capability

This area is a combination of cost and capacity in the context of capability, and to manage it, you are really looking at the systems that drive consumption.

Cost is what the resources and the consumption economically require. Capacity is what supply exists, what headroom we need, and which constraints and shortages apply. Capability is what the business and its technology need to be able to do: the features, the reliability and resilience, the customer experience, the operational outcomes. Organizations usually run cost and capacity as separate disciplines, but the two ask about the same thing. Capacity asks what we are consuming, what we will consume, and whether enough supply will exist. Cost asks what we are consuming, why we are consuming it, and what that costs. If an organization cannot explain its cost, it probably does not understand its consumption, and if it does not understand its consumption, it cannot manage capacity well.

Neither question means much without capability. You cannot responsibly manage capacity unless you know both what it costs and what people are going to use it for. Capacity has no meaning apart from capability, and cost optimization has no meaning apart from the outcome it is supposed to protect. The Cassandra cluster later in this introduction, idle by every utilization measure and indispensable to the business, is the plainest example I know.

The best picture of this I know is on the cover of Douglas Hofstadter’s Gödel, Escher, Bach. Hofstadter cut two blocks of redwood so that each casts three different letters as shadows: from one direction a block’s shadow is a G, from another an E, and from the third a B. Three shadows, one object.² Cost, capacity, and capability are shadows like that. Most organizations give each one to a different team, a split Chapter 3 examines, and each team can study its own shadow with great care and still not know the shape of the thing casting it.

The bill and the server count are outputs. What produces them is a set of systems: customers and what they do, product decisions, architecture, redundancy and recovery requirements, migrations, and now prompts, models, and agents. Those systems are what we are actually managing. I think of the work at three levels. The first is observation: what did we spend, how much capacity did we use, what does the chart say? The second is understanding: what is being consumed, where, and by whom, what is changing, and which relationships explain the change? The third is shaping: given that understanding, which interventions actually change how the system behaves? The chart is the waveform; the craft is knowing what produced it and how to influence the next one.

The record producer Rick Rubin is my favorite model for that craft. His first album credit, on LL Cool J’s debut, Radio, read “Reduced by Rick Rubin.” Years later he began recording Johnny Cash, then in his sixties, and took away the arrangements and the studio polish until all that remained was Cash’s voice and his guitar, recorded in Rubin’s living room and in Cash’s cabin. That album, American Recordings, won a Grammy.³ It is not minimalism for its own sake. Rubin’s test is to “notice how many pieces you can remove before the work you’re making ceases to be the work you’re making.”⁴ Keep the core, and let it be itself. In this work, the bill, the server count, and the utilization chart are the production. Cost isn’t the thing. Capacity isn’t the thing. Even consumption isn’t the thing. The thing is the system.

Open full-size diagram ↗

That is the promise behind this book’s title. The goal is not better reporting. The goal is enough understanding that important outcomes stop being surprising.

The demonstration was last week

The book returns to one constructed case throughout. The organization and its numbers are invented, but the pattern will be familiar to anyone who has done this work. An engineering team builds a new feature on top of an existing service and demonstrates it. The business sees an opportunity, and within a week the question is no longer whether the experiment works but how soon every customer can have it. Capacity planning enters only after the enthusiasm has become an expectation.

The organization runs a hybrid footprint, and both volumes assume it: roughly four-fifths of its modeled resource cost sits with public cloud providers, and the remainder with a private data center the company operates itself. Both sides appear in the cost discussions ahead, and they do not behave alike.

The service at the center of the case has enough capacity for its current customers, and enough for the rollout too, provided nothing fails. What it lacks is the capacity to take on the new work while keeping a promise already made to existing customers: that the service will stay up if part of it goes down. The additional capacity cannot simply be bought on the day it is needed. In the cloud it depends on quota, on the provider having the instance type available in that region, and on qualifying the new workers before they carry traffic. Quota is permission to deploy; it is not evidence that the capacity exists.⁵ In the data center it would wait on hardware. Either way, it will not be ready before the launch date.

Each group at the table sees a different problem. Finance asks why the existing resources can’t work harder. Engineering asks whether the platform can scale, and the operations team asks what happens when part of it is unavailable. The business, reasonably, wants to know how a promising demonstration turned into an infrastructure discussion. Every one of those questions is legitimate, and a meeting can spend its entire hour agreeing that each one matters and still end with exactly the shortage it started with.

Chapter 4 gives the case its first full decision treatment. The demand and reserve chapters in Volume II return to its forecasts and deadlines. The more instructive part comes afterward, when it turns out that some batch work could move and the rollout could start with a smaller group of customers. Those options change the arithmetic, and they come from people who understand the customers and the cost of delay. No utilization report would have surfaced them.

It would be better if requirements arrived before anyone promised a launch, but a capacity team that answers an urgent request with a lecture about planning discipline isn’t helping anyone; how the request should have arrived is a conversation for later. Helping does not mean agreeing to an impossible promise, either. It means understanding the problem well enough to say which conditions would have to change, what each change would cost, and who has the authority to agree to it. Kirk’s question applies: are we sure we understand the rules we are trying to satisfy, and can we change them?

Relationships are part of the information system

If the systems that drive consumption are what we manage, the next question is how to see them. The second argument of this book is that much of what drives consumption lives in how the business creates demand, and that building relationships is how much of that understanding becomes accessible. The book calls what we learn this way demand intelligence: what people know about future demand, kept with its source and its date so that we can later tell what we knew and when. I don’t mean networking so that people will attend your meeting. I mean learning enough about their work that you can recognize the difference between an idle system, a system waiting for a customer, and a system about to become a very expensive surprise.

Telemetry will tell you that a workload has been quiet. Someone on a customer team may know that several operations are about to move onto it. The billing export will show the old environment still running, but the person managing the migration can explain why it cannot be retired next month. If you only speak to engineering, you may miss the people who already have a sophisticated view of demand, expressed in units that have nothing to do with processors or requests per second.

When I ran capacity, I held a monthly meeting with each of the ten largest consumers of capacity, usually with the team’s leader or a senior technical lead: the people running the massive systems. Some teams didn’t want a monthly meeting, because their story didn’t change very often; we kept the cadence anyway. Formally, the meeting went through the plan: this was the traffic last year, this is the traffic this year, and this is what you expect to do. The most important question was never said out loud: is there anything we need to know about right now? The meeting worked a little like a therapy session for the project, a place to talk about new things the team was working on.

A lot of what mattered was in how people said things. A leader of a big search-and-indexing system said something like this: “The executives are really excited about this feature. We’re still trying to convince them how to do it in a way that’s more efficient. Here are the things they put forward.” I heard two things in that. There might be a disagreement over the implementation, and the design was heading somewhere that would cost a lot of capacity. The meetings also showed me who would be direct and who was shifting facts to fit a model. One of the most important things you can do when running capacity is identify who’s going to give you the truth without a filter.

I am an engineer, so this isn’t an argument that engineers are incapable of understanding the business. It is an argument that the information does not automatically find its way into an engineering planning system. A revenue projection or a customer pipeline may reflect plans that haven’t been announced or numbers that haven’t been approved, and the people who hold them are right to be careful about where they go. If the last tentative estimate they shared became a commitment in somebody else’s PowerPoint presentation, why would they share the next one? If admitting uncertainty earned them more reporting work and no assistance, another spreadsheet request is unlikely to improve the relationship.

You have to become useful to engineering, Finance, reliability, and the business, and you have to help them understand each other. Understand what they know, what they are still trying to find out, and what you are allowed to do with the information. A customer plan can be important without being final, and you need a way to use it without making the person who told you responsible for a guarantee they never offered. That is harder than adding a column labeled “confidence” to a spreadsheet. The confidence that matters has to be built with partners across the organization, over enough conversations that they come to trust what you will do with their numbers. That trust also gives you a better chance of hearing about the next change before it becomes a production incident.

None of this replaces the engineering or the mathematics behind a forecast, and this book spends a good deal of time on both. But a model can only work with the information it is given, and the information that changes a forecast most often arrives through relationships most engineering teams have never had a reason to build.

What changes at scale

I have had a version of the same conversation with people at several large companies. You mention a well-known book about how their company works, and the reply begins with something like “Oh, right, you read that book. Well… it’s not that simple.” Then they explain the variations in approach you could not have inferred from the book. A published account describes what one group built and learned; the trouble starts when a reader takes it for the operating manual of an entire company, and this book deserves the same caution.

What you learn is that a large company rarely manages capacity in just one way. A central strategy helps, and I am not proposing that every team invent its own method, but a company that supports different businesses, markets, technologies, and customer obligations needs some variation. It might sensibly automate most resource decisions for thousands of small applications while keeping a close planning relationship with the few customers whose requirements can determine the next major capacity purchase. A single process for both would either bury the small applications in planning meetings or leave the largest customers to an automated rule telling them that no capacity is available.

I am an ally of the FinOps community, and this book owes it a real debt. J.R. Storment and Mike Fuller’s Cloud FinOps gave this work a shared language, and the FinOps Foundation’s framework connects engineering, finance, and the business through a flexible, non-prescriptive framework whose scope now reaches past public cloud to SaaS, data centers, and AI.⁶ I see FinOps as an essential part of a larger function, one that treats capacity planning and cost management as a single discipline, because what a system costs is hard to separate from how much of it you buy, when you buy it, and what you have promised to keep running when part of it fails.

Money is the one unit every part of that function shares. Dollars add up across teams and technologies, and they make the problem legible to executives. Capacity is multidimensional by nature: processors, memory, storage, network, GPUs, tokens, requests, latency, reservations, power, and floor space, and the relationships among them. A cost figure is a lossy compression of that system. It is useful precisely because it collapses the complexity, and the collapse discards the information we need to explain why the system behaves as it does.

Visibility is one place where scale changes the answer. The FinOps principles call for cost data that is accessible and timely at every level of the organization, with engineers owning their costs from architecture design through operations. That openness is valuable when the people receiving the data can act on it. A team that has deployed a small application onto a shared platform controls neither the platform’s architecture nor its commercial arrangements, while the platform team may be able to make an improvement across thousands of applications that none of their owners could make individually. The same principles already centralize rate, commitment, and discount decisions to take advantage of economies of scale, and I would extend that reasoning to some usage decisions.⁷ Central management can be the right choice, and Chapter 3 takes up when.

Efficiency needs the same flexibility. A measured, stateless workload may respond well to a fairly simple CPU-oriented calculation. Apply that calculation to a database without understanding its data, its recovery requirements, or the maintenance it must complete, and you can make a very confident mistake. I was involved in a system where somebody judged that a very large Cassandra database was not being used, because its CPU utilization averaged around 1 to 2 percent. I had to take people aside and say that if we shut that database down based on utilization, we were going to have to shut the whole business down. It was supporting some of the most critical business capabilities around the clock. The commitment was the speed of response, not that it was being utilized all the time. Clusters like that can cost $2 million to $3 million a year to run, and this one did need to be running in Cassandra: the latency commitment and the amount of data that had to stay accessible required it, and it supported a business worth hundreds of billions of dollars.

The same calculation misleads elsewhere. An AI search service can look idle most of the day and then need substantial capacity at once to rebuild an index, perhaps so that a new product appears in results before a sale begins; a few minutes of delay can matter more than all the idle hours around it. What takes judgment is understanding why these cases differ and what evidence would justify a change in each.

The SRE and capacity-planning literature is full of hard-won experience, and I draw on it throughout. Reliability gives us one essential view of the decision. In a capacity meeting, it sits alongside the business case for a launch, the financial obligations, and the engineering work still needed to make the service useful. The participants may each be defending a promise they cannot change alone. The job is to make those promises feasible together, or to bring the conflict to someone who can change them. A capacity plan has to carry that whole decision.

Cloud-neutral, with named examples

The planning questions in this book apply across providers. Azure supplies many of the worked examples, with AWS, Google Cloud, and other services included where their arrangements help explain a choice. Naming a service lets us examine an actual measurement, deployment option, or purchasing rule. Applying the reasoning elsewhere requires checking that service’s topology, operating responsibilities, performance, and commercial terms. Similar product names do not establish equivalent capacity.

The constructed portfolio follows an organization that buys and operates computing services. The provider on the other side plans the physical supply over much longer investment and construction horizons. The book shows where the two plans meet without attempting to model a hyperscale fleet.

The examples, and what to do with them

The teaching portfolio includes a customer-facing web service, a pool of batch workers, the data platforms behind the website, an AI assistant that drafts replies for the support team, a shared platform hosting a long tail of small internal applications, and a migration with old and new environments running side by side. The figures are chosen so that we can follow the arithmetic and see how a decision in one part of the portfolio affects another.

The web service gives us a calculation we can understand without spending half the book on its implementation. Then the databases and AI work complicate it, because copying that model would conceal the very constraints we need to manage. The long tail shows how a small change repeated across many consumers can upset a plan built around the largest projects. The migration gives us a necessary improvement whose efficiency numbers get worse before they get better. None of these examples requires an unusually incompetent team. They require people doing reasonable work under conditions that keep changing.

The companion code lets you reproduce selected calculations and follow a decision from observations through forecasting, demand intelligence, supply scenarios, and review. Change the assumptions and see what happens. It is a teaching model, with explicit limits; bringing it into an operating environment would require the evidence, integration, and specialist review that the examples discuss.

The portfolio is invented; my own mistakes are not, and they belong in the discussion. I have several decades of mistakes to draw from. A book about judgment would be less useful if I only showed up after the difficult part to explain what everyone else should have done. The stories from my own work are recollections. I have left out the names of employers, customers, and colleagues, and where something happened more than once, I tell it as a pattern rather than as a single event.

Where to start

One question of positioning deserves a direct answer: when to take this job. The work assumes material the reader brings to it — years of building and running the systems whose consumption these chapters examine. People who arrive in the role as the first rung of a technology career spend their first years acquiring that material while the organization already expects their judgment. The order can be made to work, but it is the hard order. The natural order is to spend the first part of a career doing the work — engineering, operating, carrying the pager, living with the consequences of sizing and commitment decisions — and to come into cost and capacity midway through, fluent in what a workload is and what a bill is describing. The record producer I hold up as the model of craft made that order visible in his own life: Rick Rubin played in bands and learned recording from inside the work years before he produced anyone else’s records, and the judgment he is known for rests on those years. This book is written for readers at that later station, and for the organizations deciding when to place someone there.

This volume is written for people who already run cost and capacity work, or are about to inherit it. I assume you know roughly what cloud services cost, that pricing follows usage and commitment, and what a capacity shortfall looks like in practice. If you are coming to the subject cold, Cloud FinOps and the FinOps Framework are useful first stops. Here we begin with the instruments and the people around them, then ask what the work actually requires and whether an improvement is worth making.

Part I, The Decisions, separates three questions that usually arrive together: what computing costs, what capacity the systems can be relied on to provide for the capability the business needs, and who can accept the answer. Part II, Understanding the Work, follows one complete decision and the measurements and provider tools behind it. Most of its length goes to workloads whose measurements are easy to misread: AI services and their orchestration, specialized and emerging hardware, and databases. It closes with Measuring Value, because a cheaper resource count is only one part of judging a change.

If you have done this work for a while, Chapter 3’s account of authority and escalation, the AI and specialized-hardware chapters, and the discussion of operating capability may be the most useful places to begin. The closing synthesis gives you a decision record to take back to the people who will have to make it work.

Volume II carries that record into the future: forecasting, the large customers and the long tail, relationships, reserves, risks and anomalies, commitments between organizations, multiple markets, and the monthly review. It continues with Chapters 14 through 24 and provides its own recap of the portfolio. You may need a workload measurement from this volume and a negotiation from the next on the same afternoon. Keep enough structure to explain the decision, and enough curiosity to notice when that structure has stopped describing the problem. The next useful step is to discover which condition can change, and who can agree to change it.

Notes and sources

1. Star Trek II: The Wrath of Khan (Paramount Pictures, 1982), the training exercise and Kirk’s account of reprogramming it; StarTrek.com, “How Kirk Passed the Kobayashi Maru’s No Win Scenario”, film excerpt, January 8, 2022, and Jay Stobie, “Captain Kirk’s Wisest Quotes”, March 22, 2023 (first published 2021), which summarizes the reprogrammed simulation and Kirk’s disbelief in a “no-win scenario”; both checked October 3, 2026. The application to capacity management is the author’s interpretation; the manager’s remark is the author’s recollection.↩︎

2. Douglas R. Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid (New York: Basic Books, 1979), xiv. The list of illustrations describes the cover: two “trip-lets,” Hofstadter’s name for “blocks shaped in such a way that their shadows in three orthogonal directions are three different letters,” which he made from redwood, “using mainly a band saw.”↩︎

3. Andrew Romano, “Rick Rubin on Crashing Kanye’s Album in 15 Days”, Newsweek, June 27, 2013, in which Rubin recalls the Radio credit; Johnny Cash, American Recordings, produced by Rick Rubin (American Recordings, 1994), sleeve notes as transcribed by Discogs: “Recorded in Rick’s living room and Johnny Cash’s cabin,” except two songs recorded live at the Viper Room; Recording Academy, “Best Contemporary Folk Album: 1995 Winners & Nominees”. All checked October 4, 2026.↩︎

4. Rick Rubin, The Creative Act: A Way of Being (New York: Penguin Press, 2023), 242.↩︎

5. Microsoft Learn, “Check vCPU quotas”, checked October 3, 2026: Azure checks quota and capacity separately, and a deployment within quota can still fail if the region or zone lacks capacity for the requested size. Other providers differ; Chapter 6 covers quotas, stockouts, and reservations.↩︎

6. J.R. Storment and Mike Fuller, Cloud FinOps, 2nd ed. (O’Reilly, 2023); FinOps Foundation, “FinOps Framework”, checked October 3, 2026, which calls the framework “flexible and non-prescriptive” and lists scopes from public cloud to SaaS, data centers, and AI. The judgments about selective engagement are the author’s.↩︎

7. FinOps Foundation, “FinOps Principles”, checked October 3, 2026: accountability for usage “pushed to the edge,” visibility “to all levels of the organization,” and rate, commitment, and discount optimization “best centralized to take advantage of economies of scale.” The extension to some usage decisions is the author’s.↩︎