The Best Model Is a Portfolio

The Register published an interview today with Rob Ferguson, VP at Fireworks AI, a managed inference provider, and it contains a sentence worth pinning up: “The best model in the world’s not Astra. The best model in the world’s not Fable. The best model is literally a portfolio and we can prove it. You are always better routing between a few models than to do one singular model.”¹

Ferguson’s argument is about utilization, the same argument that dominates every other corner of this industry. GPUs are capital: ten thousand of them are a billion dollars, and “you don’t want those things to be sitting idle.”² Fireworks sells the alternative to owning that capital, so read the interview as a vendor making his case. The case mostly holds anyway.

What makes it interesting is not “use a cloud provider.” It is the shape of a good inference layer. Fireworks does not just forward requests to a list of models, the way an AI gateway does. Ferguson describes routers that grade the difficulty of each task and route on model affinity: “Kimi is slightly better at JavaScript than Python, things like this.”³ The claim underneath is that workload mix is heterogeneous, so a single model always pays either for capacity it does not use or for capability the work did not need. That is segmentation, and the book’s warning applies exactly: splitting traffic costs utilization, so the routing has to earn its own overhead back.⁴

The economics claim is more checkable. The real number is cost per task, not cost per token, because rework is the hidden cost. Ferguson describes a data science company where Fireworks was caching about ninety percent of tasks while the same workload on Anthropic’s Opus was caching about thirty percent: “with AI, rework is by far your biggest cost.”⁵ The token price in the price list is not the price; what you pay is tokens times the attempts a task takes to land, and no price list shows that term. His advice is to track cost per task, which is worth taking whoever you buy inference from.

On savings he claims an unusually hedged floor: “we kinda guarantee that in the case where you’re still using a combo of closed and open models, we can always guarantee that your bill will drop by half right away.”⁶ Treat the fifty percent as marketing until you have measured it on your own traffic, but the direction is believable, because a single-model setup is deliberately over-provisioned for most of its requests.

Two caveats easy to skip: frontier providers are not transparent about cache hit rates, so your measured cost per task is the only honest comparison; and models memorize, so if a provider trains on your data, that data has value and you should be negotiating it.⁷

None of this requires buying from Fireworks. Know your traffic mix, measure cost per task, and treat “which model” as a routing question with a measurable answer instead of a loyalty question.

Sources