Ask again, do you really need to buy a GPU?

The conversation that starts a GPU purchase rarely starts with a workload. It starts with a meeting. The chief executive asks, “What is our AI strategy?” and somewhere in the answer the word “strategy” gets translated into hardware. A request arrives for H200s. Not because a benchmark failed, and not because a hosted model was measured and found wanting, but because owning GPUs feels like having an AI strategy. It makes the team feel relevant. It feels like the next step.

It isn’t the next step

The feeling is understandable, but the sequence is backwards. The H200 is the last step of a process, not the first one. The steps before it are a measurement, an experiment, and a cost comparison, and none of them require owning anything.

This is the argument of Chapter 10, “Specialized Hardware”, which now carries a new section making the point directly: the serving experiment that justifies a GPU fleet does not need a GPU fleet. Before any purchase, the experiment runs on capacity someone else already owns.1 Three services cover it, in the order I would investigate them:

• NDIF + NNsight, a free, NSF-supported research service (nnsight.net), runs model-intervention code remotely against models already deployed on shared GPUs, including Llama-class models at 70B and 405B. If the question is “what is the model doing internally,” this is where the answer starts, and the price is a registration.2
• Runpod GPU Pods (docs.runpod.io) is a rented GPU machine with SSH and Jupyter access. More setup, but strong control over checkpoints and experimental settings.3
• Modal (modal.com) runs custom Python and PyTorch remotely, on one GPU or several, with the environment defined in code.4

There is one more reason the H200 can wait. Unless the work needs real isolation, or it uses techniques that reach into the model itself, like activation steering or custom training, most teams are calling an API with extra steps. The shared services above already do that job. Working with hidden activations, patching layers, or training your own weights: that is when a machine of your own, rented or owned, starts to earn its place. An H200 bought to feel relevant will spend most of its life idle.

Prove it, then price it

The point of starting there is not that renting is virtuous. It is that renting makes the proof cheap and the purchase honest. Runpod bills pods by the second. Modal bills GPU, CPU and memory by the second, and charges nothing while the code is idle. The current list prices in the chapter: an H100 at $0.001097 per second on Modal, which is $3.95 per GPU-hour, and roughly $2.89 to $3.49 per hour on Runpod.45 Against a purchase price of roughly $25,000 to $30,000 for one H100 card, a month of experimentation costs less than the sales tax on the machine.

So the sequence, in the order the book gives it:

1. Prove the workload. Run the serving experiment, goodput against latency objectives at the real prompt lengths, on a rented GPU or a research service. If the workload turns out to fit a hosted model, that is a finding, not a failure.
2. Do the cost comparison. Take the measured configuration and compare it against a per-second or per-hour resource, then against commitment pricing, then against ownership, with power, cooling, staffing, failure allowance and refresh in the ownership column.

If the evidence survives all of that, the purchase is an engineering decision, and this book will help you size it. If it doesn’t survive, you found out for the price of a meter reading instead of a purchase order.

The next time the CEO conversation contains the words “What is our AI strategy?”, try answering with one: “We’re proving it.”

Sources

1. Scale Without Surprise, Volume I, Chapter 10, “Specialized Hardware” — new section “An experiment on shared hardware, before the fleet,” with the three services, published rates and the cost comparison: https://withoutsurprise.com/chapter-10 (section added October 14, 2026).

2. NDIF / NNsight, “Remote Execution”: https://nnsight.net/features/15_remote_execution/ — free account registration; models served on shared GPU pods; a one-hour limit per remote request. Accessed October 14, 2026.

3. Runpod documentation, “Connection options” and “Pricing”: https://docs.runpod.io/pods/connect-to-a-pod and https://docs.runpod.io/pods/pricing — pods billed by the second; storage billed while stopped; default spend limit of $80/hour. Accessed October 14, 2026.

4. Modal documentation and pricing, “GPU acceleration”: https://modal.com/docs/guide/gpu and https://modal.com/pricing — H100 at $0.001097/second ($3.95/GPU-hour); billing per second for GPU, CPU and memory; T4 through B300 in the catalog. Accessed October 14, 2026.

5. Runpod, “NVIDIA H100 Price”: https://www.runpod.io/articles/guides/nvidia-h100 — authorized-reseller pricing of approximately $25,000–$30,000 for a standalone H100 PCIe card. Accessed October 14, 2026; reseller estimates, not a quotation.