Abstract:
Can you believe that OpenAI’s first chip production actually defeated the entire NVIDIA series? ! Just now, OpenAI's first self-developed chip, Jalapeño, handed over the first batch of measured results - AI inference performance, completely surpassing NVIDIA GB200 and GB300.
Take off directly at the measured speed!
Jalapeño can spit out 1,459 tokens in one second, while Nvidia GB200 can only spit out 535 tokens, which is less than half.
It only takes 1.65 seconds for "Pepper" to complete a task, but it takes nearly 6 seconds for GB300, a full 3.6 times difference.
The most cruel thing is that even if GB300 is forced to its fastest decoding speed, Jalapeño's throughput is still 104.3 times higher than it.
The nominal power consumption of Jalapeño is only 700W, while that of GB300 is 1400W, which is exactly half.
Famous semiconductor analyst Dylan Patel even said harshly, "It's not just Blackwell that has been overthrown, even Nvidia's Rubin chip has been surpassed"!
For a time, the entire AI circle exploded. Netizens marveled that OpenAI was so crazy that Nvidia was also defeated.
Ultraman's statement was domineering yet restrained, "We made a chip, it's ridiculously fast"!
01
A "chili pepper" overturned Nvidia's flagship
This is not OpenAI talking about itself.
SemiAnalysis went into the laboratory for actual measurements, and the conclusion they wrote is——
Jalapeño defeated every Nvidia, AMD, and Google chip they had tested before.
How big is the difference? Nvidia’s latest Vera Rubin consumes enough electricity to feed nearly three Jalapeños.
During the test, OpenAI selected three public models to run: GPT-OSS 120B, and a 670B and 1T model.
This round of testing covers MoE architectures from medium to trillion parameters. The overall results are as follows:
AI workload per watt increased by 1.5–1.9 times
End-to-end latency reduced by 1.7–3.6 times
2.1–4.1x performance improvement on highly interactive workloads
The 1459 Token/s mentioned earlier were measured on GPT-OSS 120B.
Converted to the word interval, there is only 0.69 milliseconds between the two Tokens.
A person normally reads about five or six words in one second, but it spits out 1,459 words in one second. The eyes cannot keep up, and the screen cannot read them.
Green is Jalapeño, blue is the strongest available. The three models are 2.7 times, 4.1 times and 3.8 times faster respectively
The four-second gap between 1.65 seconds and 5.99 seconds can be tolerated in chat, but it is another matter when it is used on an agent, because a task requires dozens of consecutive steps, so the difference needs to be multiplied.
Then it was said that the whole network was rotating 104.3 times.
What it exactly means is that pushing GB300 to its fastest decoding speed, which is 169 Tokens per second, at that point, Jalapeño's throughput is 104.3 times that of it.
This trend is true for the three models, with GPT-OSS reaching 53.7 times and the 1T model reaching 56.1 times.
For the same model, as the required word production speed increases, the leading margin increases from 1.7 times to 104.3 times
In other words, the place where the opponent reaches the limit and begins to pant is exactly where it is still cruising calmly.
If the peak values are calculated based on their respective best working points, the gap will be much milder, with the three models being 1.9 times, 1.7 times and 1.5 times respectively.
What really creates distance is the high-interaction scene
The horizontal axis of the left picture is the word output speed, and the horizontal axis of the right picture is the delay of the complete request. The green Jalapeño presses the blue GB200 on the entire curve
SemiAnalysis takes into account power supply, heat dissipation, and network, and then spreads it to each chip.
As a result, Jalapeño has 1.125 kilowatts per unit, GB200 has 1.87 kilowatts, and Vera Rubin has 3.3 kilowatts.
When running GPT-OSS at this caliber, Jalapeño spits out about 53 million tokens per megawatt per second, and GB200 NVL72 only spits out about 10 million.
The purple one is Jalapeño, the horizontal axis is the interaction speed, and the vertical axis is the number of tokens spit out per megawatt per second
In terms of cost, the total cost of ownership per chip hour is $1.56 for Jalapeño, which is almost the same as $1.55 for H100, while Vera Rubin is $3.61.
Green is Vera Rubin, purple is Jalapeño. After 200 tokens per second, purple went all the way to a place that was out of reach of green
The most terrible thing is that these results are not Jalapeño's full strength.
It does not even enable speculative decoding and token prediction, nor does it perform separate deployment of prefill and decode. However, every other chip in the picture is running its own optimal configuration, and none of the optimizations that should be enabled are left out.
SemiAnalysis estimates that speculative decoding alone can reduce the cost per Token by more than 2/3.
A Jalapeño can also run the "Doom Slayer"
02
In nine months, GPT‑Astra worked all the way to the tape-out room
For such a thing, it only took nine months from design to tape-out of OpenAI. In comparison, a common timeframe in the industry is 18-36 months.
Everyone who has done ASIC knows that the most grueling job in this cycle is verification. The simulation has to be run over and over again, and if you change a place, you have to start all over again.
The reason why OpenAI can "cheat" is because it invites its most powerful model into the tape-out room——
The model that helped develop Jalapeño is called GPT-Astra, which is the GPT-6 that has been rumored by the outside world!
Throughout the entire process, GPT-Astra basically became the team’s strongest support.
It helps explore implementation solutions, squeezes the whole "design-measurement-verification" circle to death, and also micro-manipulates arithmetic circuits.
To what extent is micromanaging done? The figures dug by SemiAnalysis are that AI-assisted design reduces the SIMD unit area by 8% and the matrix engine area by 10%.
At the same time, these modules are better than the first version in terms of timing and power consumption.
The computing die is in the center, with three HBM4s on each side, and the top one is the I/O chip
The main calculation die is about 840 square millimeters, and the mask limit of the EUV lithography machine is 858 square millimeters. This piece of silicon is only 18 square millimeters away from the physical ceiling.
It can be said that the area where the photolithography machine can be used for drawing is basically occupied, leaving nothing at all.
The pink row is Jalapeño, and the three rightmost columns are HBM bandwidth per watt, FLOPs per watt and computing power-to-bandwidth ratio
In terms of HBM bandwidth per watt, Jalapeño is 22, second place Rubin is 11.1, and GB300 is only 5.71. It is exactly twice as high as the second place.
The FLOPs per watt is 19.1, and the Rubin’s is 19.4. The two are almost tied, but the Rubin burns more than 1,800 watts, while the Jalapeño only burns 700 watts.
This is just running with A0 steps. B0 is already in the fab, and its performance per watt is about 25% higher than A0.
In this way, relying on the powerful combination of Codex and GPT-Astra, OpenAI transformed three open source models that were not originally planned into a high-performance state in just two months.
What’s even more frightening is that in the most performance-hungry “attention” and MoE modules, the code typed by AI runs 1.5 to 1.8 times faster than top human experts.
AI designs the chip, the chip runs AI, and the AI goes back to optimize the AI on the chip.
This flywheel, OpenAI is already spinning.
CUDA moat, dead?
NVIDIA's deepest moat has always been CUDA.
The software stack accumulated over the past two decades has cultivated the muscle memory of engineers around the world. Every time the hardware is changed, they have to tear it down and start all over again.
SemiAnalysis made a very heavy judgment in the article, saying that CUDA's moat may be dead.
The reason is speed.
They also added a more heart-wrenching comparison: Nvidia’s Rubin actually completed CoWoS tapeout one month earlier than Jalapeño.
But until now, the only Rubin results that the outside world can see are the data from CoreWeave engineering samples. Nvidia does not put third parties into the laboratory for random testing like OpenAI.
So the problem lies in the speed at which the software starts. There is nothing wrong with NVIDIA's hardware itself.
Starting from scratch even gives OpenAI an advantage. It does not need to carry historical baggage, and the architecture can be drawn completely on a blank sheet of paper.
The last sentence of SemiAnalysis is very graphic——
They say that OpenAI models such as GPT-5.6 Sol running on NVIDIA GPUs have been used to design a chip that truly threatens the CUDA moat. NVIDIA's own GPUs are spawning their own "successors" in real time.
03
10 gigawatts, this is just the beginning
Jalapeño is just the first shot on a front.
In October last year, OpenAI and Broadcom made a big move and signed a large cooperation order for a 10GW custom accelerator.
Ultraman and Broadcom CEO Chen Fuyang showed off the Jalapeño wafer, with the nameplate reading "Jalapeño Intelligence Processor"
The 10GW level is almost equivalent to ten nuclear power plants operating at full capacity.
The CPU main cabinet on the left, the ASIC cabinet on the right, 128 chips in one cabinet, the total of the two cabinets is about 160 kilowatts
By the way, the tray that holds the CPU is called Katsu, the chip is called Vindaloo Indian curry, and the switch is called Chana chickpeas. From Japanese food to spicy Indian food, these people really want you to remember this system.
And Jalapeño is only the first generation on the roadmap, OpenAI clearly stated in the report——
Gen2 is already under deep development, and Gen3 is also taking shape.
Mass production will not gradually ramp up until 2027, and the next milestone is 100 megawatts.
However, just today, it was revealed that OpenAI’s data center manager, Chris Malone, is gone!
Malone is the head of Stargate.
Now, the IPO is getting closer and closer. At this juncture, the loss of the key commander of the computing infrastructure makes people wonder about the undercurrent behind OpenAI.
Of course, one Jalapeño is not enough to overthrow Nvidia.
But the real danger signal is that OpenAI has twisted models, chips and software into an accelerating flywheel.
Today’s Jalapeño is only the first generation, followed by Gen2, Gen3, and a 10GW computing power landscape. Even if the core executives leave, it cannot stop this route from continuing to move forward.
Nvidia still sits on the throne.
Only this time, the replacement didn't line up outside the door, it had already run into the computer room.
Nvidia’s rivals finally began to build their own Nvidia.
Comments