OpenAI and Broadcom taped out Jalapeño, OpenAI's first custom inference chip, in nine months...
OpenAI and Broadcom have taped out Jalapeño, OpenAI's first custom inference chip, with the full design cycle completed in nine months and OpenAI's own models contributing to the design process. For anyone tracking where the economics of large language models actually get decided, this is one of the more important silicon stories of the year, and it has very little to do with replacing Nvidia.
The framing matters. Jalapeño is purpose built for inference, not training. Inference is the workload that scales linearly with users, every ChatGPT turn, every Codex completion, every step an agent takes. Training is a fixed capital cost. Inference is a per query cost that compounds with every product success. OpenAI claims performance that is substantially better than state of the art, which in inference terms means tokens per second per watt. When you are power constrained across multiple data centers, and most hyperscalers now are, that ratio is the only number that matters. It determines how many users you can serve before you run out of grid capacity.
The nine month timeline is the detail builders should sit with. Traditional silicon design cycles run two to three years from architecture to tapeout. Compressing that to nine months suggests OpenAI's models were used inside the RTL design and verification loop in a serious way, not as a marketing flourish. If that holds up, Jalapeño is also a proof point for AI assisted chip design as a real production workflow rather than a research demo. That has implications well beyond OpenAI, because every other lab and hyperscaler is watching the same playbook.
The strategic consequence is concrete. Over the next 18 months, the cost per token curve gets set by whoever owns the silicon, not whoever owns the model weights. Hosted inference margins get squeezed from the bottom, and providers without their own accelerators feel it first. This is why Google has TPUs, why Amazon has Trainium and Inferentia, why Meta has MTIA, and why Microsoft has Maia. OpenAI was the last major lab without a custom inference path, and Jalapeño closes that gap.
What is worth watching from here is the deployment cadence. A tapeout is not a production fleet. The interesting questions are when Jalapeño actually serves real ChatGPT traffic at scale, what fraction of OpenAI's inference moves off Nvidia hardware over the next year, and whether the nine month design cycle becomes a repeatable pattern or a one time outlier. If it repeats, the competitive surface of the model layer starts looking a lot more like the competitive surface of the chip layer, and the winners will be the ones who can iterate on both at the same speed.
Originally posted on LinkedIn.