By Christopher Mims

No one could accuse OpenAI Chief Executive Officer Sam Altman of thinking small. He once proposed building a solar array around our sun to power future AIs.

His company's new deal with Broadcom to make custom chips tailored to the needs of OpenAI's future customers might not be out of this world, but it's certainly audacious.

Altman has said that to deliver the artificial-intelligence services consumers want, his company's data centers will need at least one AI-specific chip per user. In other words, billions of chips.

Experts echo this notion. If AI takes over all the tasks we've been promised it will, the world will need as many AI microchips as it currently has conventional microchips, says Ali Farhadi, CEO of the nonprofit AI research organization Allen Institute for AI.

Nvidia is still the go-to when companies are building data centers to train their AIs. But custom chips can make the process of delivering AI -- what's called inference -- faster and cheaper. That could help OpenAI save money in its push to reach profitability, something it's far from achieving at present.

OpenAI's recent deals with Broadcom and Nvidia are the peanut butter and chocolate of the AI world. Both are necessary for OpenAI to achieve its goals of training the world's most capable models -- on Nvidia's chips -- then delivering their output without breaking the bank -- on Broadcom's custom chips.

DIY silicon

Amazon.com and Google have long matched custom silicon to complex software to power cloud computing, and both also design their own custom chips for training and delivering AI. Meta and Microsoft are in the early stages of attempting their own custom AI chips.

Usually with computers, software developers have to write programs that are tailored to the existing hardware, and specifically microchips, on which they run. When a company starts designing its own chips, as Apple did back in the mid-2000s, they have a chance to match chips and software more closely. That's why the iPhone is fast and power-efficient. For OpenAI, more-efficient chips mean spending less on electricity to deliver its AI to customers.

Broadcom is giving OpenAI a means to remix the typical AI-chip recipe, says Jordan Nanos, a former engineer at Hewlett Packard Enterprise and member of the technical staff at semiconductor and AI research firm SemiAnalysis.

Nvidia intends its chips to be flexible. They are high-powered and versatile in order to suit any number of AI applications. And when it comes to training models, industry leaders agree it's the top choice: Nvidia's market share for training models is upward of 70%, based on several estimates.

When delivering an AI model to users through inference, the hardware needs are different than when training that model. This creates an opportunity for AI companies, which can use chips customized to run their specific applications faster and more efficiently.

During the inference phase, OpenAI's models run best on chips that support large amounts of what's known as high-bandwidth memory, says Nanos. Earlier this month, the company announced a partnership with the two leading companies that make such memory, Samsung and SK Hynix.

This dependency on high-bandwidth memory is common among AI models but not universal. OpenAI's desire for chips of this kind is in some ways specific to the models it makes, and the applications it believes its future customers will be using in the greatest amount.

OpenAI doesn't disclose what those future applications are, but during the company's announcement, Altman said that its Pulse product, which uses AI agents to scour the internet and brief a user on topics of interest every morning, requires so much computing power that the company has to limit its rollout solely to those who pay $200 a month for its Pro tier.

Another factor connects chip design to energy usage, and that's "sparsity." Earlier models were "dense": Every time they received a prompt, a substantial portion of the nodes within their neural networks had to be activated to answer it. This corresponded to a proportionally large amount of computing.

Newer models divide up the expertise of a model, so various sections of the neural net are best at a given kind of query. By activating fewer "experts," less compute is needed. Early models might activate a quarter of their neural network to answer a typical question; the newest ones activate a fraction of a percent. A chip designed specifically for a model that works in this way can run far more efficiently.

AI supercomputers

While AI chips are at the center of OpenAI's deal with Broadcom, the bigger picture is about building entire, gigawatt-scale AI supercomputers. That means Broadcom is also on the hook to deliver the networking chips, cables and optical interconnects required to tie it all together.

Altman has said that OpenAI's total AI compute amounts to 2 gigawatts, although not all in one place. The Broadcom deal involves rolling out up to 10 gigawatts of AI systems jointly developed by OpenAI and Broadcom by 2030. That's on top of the 16 gigawatts of deals announced in the last three weeks, with AMD and Nvidia -- a scale of computing power that boggles the mind.

In total, these commitments could require close to a trillion dollars in investment and two New York City's worth of electricity.

OpenAI isn't the only one hoovering up chips and energy. In September, xAI announced that its Memphis AI Colossus supercomputer had reached 1.21 gigawatts in capacity. Meta has received approval for 2.3 gigawatts of power generation at its under-construction Louisiana AI supercomputer, code-named Hyperion. In July CEO Mark Zuckerberg declared it would ultimately be 5 gigawatts in size.

Altman called building AI infrastructure "the biggest joint industrial project in history," in a podcast announcing the Broadcom deal. And yet, he added, this deal is "a drop in the bucket compared to where we need to go."

Yet another goal of this deal is diversifying OpenAI's suppliers. OpenAI's Stargate site to be built by Oracle in Abilene, Texas, is for training, so it is likely to consist almost entirely of Nvidia chips, which remain the industry standard for this application. OpenAI has also pledged to buy from AMD, but these chips are likely to be primarily for inference.

"OpenAI is looking quite far into the future, and trying to make sure they have access to enough supply of chips," says Nanos.

Write to Christopher Mims at christopher.mims@wsj.com

(END) Dow Jones Newswires

October 17, 2025 09:00 ET (13:00 GMT)

Copyright (c) 2025 Dow Jones & Company, Inc.