To explain how it works, let us compare OpenAI's and Nvidia's approaches. Nvidia's standard system-level decomposition is primarily two phases: prefill and decode. Prefill is generally compute-bound (which is why Nvidia tried to assign Rubin CPX with GDDR7 for this one), while decode is generally memory-bound (which is why Nvidia wants to keep GPUs with HBM for this). By contrast, OpenAI breaks the process into three phases: prefill (compute-bound), draft (latency-bound), and verification (bandwidth-bound).
In theory, OpenAI could have assigned each phase to a different type of specialized accelerator. However, the amount of prefill, drafting, and verification changes depending on the model, context length, token efficiency, and software algorithms. As a result, it is hard to predict the number of processors that must be deployed, so it's inevitable that some phases would sit idle. To that end, it makes more sense to develop one balanced ASIC that can do everything.
As a bonus, a universal inference accelerator does not need to move increasingly large KV cache to its peers, unlike highly specialized ASICs, which ultimately means less power consumed.
Despite Jalapeño's exceptionally short development cycle, OpenAI says that most of the processor was designed from scratch rather than assembled from existing Broadcom accelerator IP. OpenAI's Richard Ho said the compute die reuses some interface IP, but most of its Register Transfer Level (RTL) was newly written using XLS and Verilog. Meanwhile, development moved remarkably quickly: initial RTL work began in February 2025, the design taped out in November, first silicon arrived in May 2026, and OpenAI had Codex running on Jalapeño that same month.
One reason OpenAI was able to move so fast with its development was its extensive use of AI to assist and optimize the design . More than half of the core was written using the XLS hardware language and compiler infrastructure, while OpenAI's own AI models searched for ways to improve power, performance, and area (PPA). Compared with human 'baseline' designs, OpenAI reports improvements of 56% for a BF16 multiplier, 21% for an FP4 dot-product block, and 10% for an FP32 accumulator, along with 10% and 8% area reductions for the matrix and SIMD units. The company says that AI-assisted optimization even helped squeeze circuitry into a floorplan block that otherwise would not have fit, though OpenAI remains tight-lipped about what circuitry it was.
Jalapeño is OpenAI's first, but not the last, attempt to develop custom inference hardware. The company says its 2 nd Generation already well into development and heading toward tape out, while Richard Ho said during the presentation that Gen 3 is already 'operational,' even though his slide said 'planned.'
Because Jalapeño features its unique spatial and sliced architecture, it is programmed differently from Nvidia's CUDA GPUs. OpenAI says Jalapeño can be programmed using a low-level programming environment in the open-source Triton ecosystem. Unlike the latest versions of CUDA , which ensure that its code runs on all Nvidia GPUs and aligns with the tensor-heavy execution model of Blackwell processors and their successors, Jalapeño gives software more explicit control over where data and computation are placed.
Each core has fast access to its local portion of HBM; these cores are interconnected using an ultra-fast network, so the software must be able to determine where tensors are physically located and how they are distributed across the chip. This makes programming the spatial architecture more complicated, so OpenAI also uses AI to find efficient data placement, scheduling, and communication patterns and to optimize kernels for the hardware.
Meanwhile, optimal mapping is architecture-dependent, so once OpenAI changes the number of cores, local-memory organization, collective-network topology/bandwidth, or compute resources in next generations of its accelerators, the old placement and scheduling may no longer be optimal and will require AI tools to perform hardware-specific optimizations again. By contrast, software written for Blackwell will work on Rubin and then Feynman without modifications.
But how good is OpenAI's software stack compared to CUDA? Apparently, good enough, based on performance results published by the company.
Instead of comparing peak performance numbers, OpenAI used SemiAnalysis ' InferenceX benchmark to compare Jalapeño and Nvidia's GB200/GB300 across their complete latency-versus-throughput curves. The company measured how many tokens each system could deliver at comparable user-perceived latency and normalized the results by package power — 700W for Jalapeño, 1,200W for GB200, and 1,400W for GB300 — meaning that while Nvidia's hardware can lead in terms of absolute performance, OpenAI's accelerator leads in efficiency.
(Image credit: OpenAI) (Image credit: OpenAI) (Image credit: OpenAI) (Image credit: OpenAI) (Image credit: OpenAI) (Image credit: OpenAI) (Image credit: OpenAI) (Image credit: OpenAI) (Image credit: OpenAI) Image 1 of 9 View Original
Compared to Nvidia GB200/GB300, Jalapeño delivers:
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/hot-chips-2026-openais-jalapeno-ai-asic-unpacked-accelerator-developed-using-ai-achieves-efficiency-and-throughput-gains-against-power-hungry-blackwell#main
- https://www.tomshardware.com/my-account
- Hot Chips 2026: Arm details AGI server CPU with two 70-core N3P chiplets — touts 2 TB/s UCIe fabric link and 12-channel memory controller
- Bill Gates calls for some jobs to be ‘Human Reserved,’ suggests taxing AI tokens and robots — billionaire says that ‘AI era will be one of the most turbulent ti
- Hot Chips 2026: Fujitsu's Monaka CPU stacks its entire cache on a separate 5nm die and narrows to 256-bit SVE2 — 350W and 500W SKUs due in 2027
- Google clamps down on Android app RAM usage amid AI memory crisis — developers have until February 2027 to adapt to new memory-optimizing rules
- Hot Chips 2026: High Bandwidth Flash promises massive bandwidth and capacity, but its usability is extremely limited — new memory format strikes a balance betwe
Informational only. No financial advice. Do your own research.