OpenAI Launches Jalapeño: Its First Custom Chip

OpenAI and its first custom chip: Jalapeño

OpenAI has just taken a bold step into the custom chip world, revealing Jalapeño, its first inference processor built in partnership with Broadcom. This chip was designed specifically to meet the unique needs of OpenAI's inference systems. And the most interesting part? OpenAI's own AI models helped develop Jalapeño. Although the chip is still in testing phases, early results already indicate significantly better performance-per-watt compared to the most advanced alternatives available today.

The official announcement of this partnership came in October, but rumors of OpenAI's plans to build its own chips had been circulating for some time. The idea is to reduce the company's dependency on Nvidia GPUs. Google and Amazon have already trodden this path with their own custom chips, known as "AI accelerators." These chips are designed to accelerate machine learning workloads. Greg Brockman, president of OpenAI, explained this approach in an episode of the company's podcast shortly after the Broadcom partnership was announced. He highlighted that OpenAI has a deep understanding of the workloads and is seeking to accelerate what is possible to do.

Jalapeño: the chip that could change the game

Jalapeño was designed specifically for inference, which is the process of running pre-built AI models in response to user commands. In practice, this means the chip is optimized to run real-time coding models with low operating costs. Although more intensive tasks, such as pre-training, are still expected to rely on Nvidia hardware, even minor reductions in inference costs can significantly improve OpenAI's financial results.

This shift toward custom chips is a crucial step in AI economics and is expected to occur at every level of the tech stack. OpenAI is already building products like Codex and the models that power them, along with data centers to run these models. By entering the field of tailor-made chips, the company can optimize every layer of its stack around a single goal: to make its models faster, more reliable, and accessible for users.

The infrastructure behind the models

OpenAI is not just developing cutting-edge models or building products on top of them. It is designing the infrastructure that sustains them: chip architecture, kernels, memory systems, networks, scheduling systems, deployment systems, and the product experience. This means that, because OpenAI operates across the entire stack, every layer can be optimized for the same goal. And that goal is clear: making its models faster, more reliable, and more accessible to users.

The creation of Jalapeño marks a new chapter in OpenAI's history. By developing its own infrastructure, the company not only increases its autonomy but also opens new possibilities for the future of artificial intelligence. The question now is: what will be OpenAI's next step on this path of innovation?