OpenAI Publishes First Jalapeño Benchmarks: 1.5-1.9x More Work Per Watt, 1.7-3.6x Lower Latency

On August 25, OpenAI published measured results for Jalapeño, its first custom inference chip¹. The company says it has been testing the chip and the system built around it since announcing it, and that the results show gains in both AI work per unit of power and response speed.

The figures span three open-weight models. Across GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 1T, OpenAI reports 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems, with 2.1 to 4.1 times higher performance for highly interactive workloads¹. These are measurements OpenAI ran itself, not third-party verification.

What the “1.9x” Is Actually Comparing

The first thing to note about this announcement is that OpenAI has chosen to measure performance per unit of power rather than per chip. The company states that choice explicitly: performance is sometimes reported per chip, but it believes the more useful standard is performance per unit of power¹.

The measurements were run on InferenceX, a public benchmark from SemiAnalysis that the company describes as measuring the full process of serving an AI request¹. OpenAI says it compared Jalapeño against leading commercially available AI systems across the tested operating range, from high-throughput serving to highly interactive, low-latency use¹.

The normalization method is stated as well. Results were normalized using each accelerator’s published chip power rating, with Jalapeño rated at 700 watts — though its measured sustained power stayed at or below 550 watts on the workloads tested¹.

The appendix carries more specific test conditions. The GPT-OSS 120B chart is labeled “InferenceX · GPT‑OSS‑120B · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB200 1,200 W,” and the DeepSeek R1 and Kimi K2.5 charts list GB300 at 1,400 W¹. What that shows is systems with different power ratings being compared after each is divided by its own published power figure. Either way, the published numbers are per unit of power; they do not translate into “this many times faster per chip.”

Per-model figures appear in the appendix too. For GPT-OSS 120B, peak throughput per kilowatt is about 1.9x (85,448 vs. 44,960 mixed/kW) and end-to-end latency about 1.7x lower (1.03s vs. 1.80s)¹. On Kimi K2.5, the largest public model tested, OpenAI reports approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency¹.

The company also says the advantage widened further on frontier OpenAI models — but that comes from internal testing, not from published benchmark results¹.

Why Latency Compounds for Agents

OpenAI frames its evaluation approach as measuring “at a matched user experience”: how much useful AI work each system can complete per unit of power while meeting the latency that customers and interactive agents require¹.

The reason it emphasizes latency is that agents need to complete many steps in sequence, so delays compound across an entire task¹. A single response that is one second slower becomes twenty seconds across a twenty-step agent run. For anyone putting agents into a workflow, that accumulation can matter before model quality does.

The architectural explanation follows the same line. Language-model inference splits into prefill, which processes the prompt and is compute-intensive, and decode, which generates the response token by token and is constrained more by memory bandwidth¹. OpenAI says Jalapeño was designed to minimize data movement and communication delays, so model state — including the KV cache used while generating a response — can be explicitly placed and kept local while the system activates the right combination of compute, memory and networking for each inference phase¹.

Designed With AI, Designed So AI Can Program It

The development account is specific. OpenAI says AI played a direct role in Jalapeño’s development, letting the team move from initial design to tapeout in nine months by exploring implementations, shortening design, measurement and verification loops, and iterating continuously on model workloads¹.

The chip itself was designed as a clear, predictable programming target for both humans and AI. Using Codex with GPT‑Astra, the team brought three open-weight models that were not part of Jalapeño’s original production plan to high performance within two months¹.

One caveat comes from the company itself. For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than existing human-expert-written implementations — but OpenAI notes that those figures apply to the selected blocks, not the full model¹.

Deployment Comes Later, and NVIDIA Stays

The timing matters for practical purposes. OpenAI says it plans to begin deploying Jalapeño within its own compute infrastructure by the end of the year, with Gen 2 deep in development and Gen 3 taking shape¹. For now it is continuing production qualification, maturing the software, preparing to operate Jalapeño at scale, and validating performance across more models¹.

At the same time, the company states that it will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference workloads¹. This is not a story about first-party silicon replacing external supply.

On what the efficiency gains enable, OpenAI lists three items: ultra-fast-mode inference at efficiencies previously available only in fast mode, fast-mode inference at efficiencies previously available only in batched mode, and higher efficiency for batched-mode inference¹. It frames this as producing more useful work from the same power and hardware, which helps it serve more demand and lower the cost of delivering a successful result — creating room, in its words, for useful work and revenue to grow faster than the cost to serve¹.

The Summer Inference Speed Became the Axis

This announcement sits in a run of recent moves. Cerebras announced on August 13 that it supplies the inference behind the OpenAI API’s high-speed Ultrafast tier, and OpenAI itself posted a temporary price cut for GPT-5.6 Sol on August 21. On the hardware side, NVIDIA announced full production of the Groq 3 LPX inference accelerator on August 24.

“How much can the same power serve, and how long does it keep you waiting” is now discussed alongside “how capable is the model.” That said, Jalapeño has not been deployed yet, and nothing in this announcement connects the recent price cut to the chip.

For anyone running the API in production, nothing changes today. What this announcement makes possible is a plan: once deployment starts later this year, check whether response times and pricing actually shift — against your own usage records rather than published figures.

Sources

1. Jalapeño’s first results show industry-leading speed and efficiency in AI inference - OpenAI official announcement (August 25, 2026)