Coinbase just cut its AI spend nearly in half while token volume kept climbing, and CEO Bria...
Coinbase reported this week that it nearly halved its AI spend even as token volume kept climbing, and CEO Brian Armstrong credited the savings to routing, caching, and leaner context rather than switching to a smarter model. It is a small data point on its own, but it lines up with a larger shift in how the economics of AI inference are settling out.
The supporting signals are stacking up fast. Hugging Face crossed 100 million dollars in annual recurring revenue, largely on the back of Chinese open weight models that developers are now treating as serious production options. JPMorgan has been telling clients that most future tokens will not come from frontier labs at all. Amazon Bedrock is now hosting roughly half a dozen open models at a fraction of GPT class pricing, which gives enterprises a one click path to swap providers without renegotiating contracts. Taken together, the market is pricing raw intelligence as a commodity, and the premium for being the smartest model in the room is shrinking quickly.
The technical shift underneath is what makes this stick. A year ago, "call GPT 4" was a defensible architecture choice that a senior engineer could justify in a design review. Today it is a line item that finance will question by the next quarter. The teams winning on cost are building a routing layer that classifies each incoming request, sends the easy 70 percent of traffic to a small open model, reserves frontier calls for the genuinely hard tail, caches aggressively on the prompt prefix so repeated system instructions are not re billed every turn, and trims context before it ever hits the model. None of these techniques are new individually. What is new is that the savings are now large enough to fund a dedicated platform team.
If you are a builder, the work has moved up the stack. Model choice is now a runtime decision, not a procurement one, which means your infrastructure has to assume the underlying provider will change. Your moat is the router, the evaluation harness that catches quality regressions, and the observability layer that proves a model swap was safe before it ships to users.
The interesting thing to watch next is whether the big inference providers try to absorb this routing layer themselves, offering it as a managed service, or whether a new category of independent tooling emerges to sit between applications and the model zoo. Either way, the center of gravity in applied AI is moving away from the model weights and toward the systems that decide which weights to call, when, and how often.
Originally posted on LinkedIn.