Picture a world-class chef working in a kitchen with no countertops. She can chop faster than anyone alive, but every ingredient has to be handed to her one item at a time by someone jogging in from a pantry down the hall. Her knife skills stop mattering pretty quickly. The jogging is the bottleneck.
That is roughly the situation inside every AI accelerator you have read about. We talk about these chips as though the processor is the whole story, but the processor spends a shocking amount of its life waiting. Sitting right beside it, sharing the same slab of packaging, is the part that actually decides how fast things go: the memory.
Two chips in a trench coat
When someone says “AI chip,” they usually mean one silicon die that does the math. In practice, a modern accelerator is a small neighborhood. There are one or more large logic dies doing the calculations, and stacked next to them are towers of memory called HBM, short for high bandwidth memory. All of it gets mounted onto a specialized base layer, and the whole assembly is sold as a single product with a single name.
Meta’s third-generation in-house accelerator, MTIA, is a good example of how the recipe reads in 2026. It is expected to be built on TSMC’s N3P process, likely carrying over 100 billion transistors, paired with HBM memory. That last part is not a footnote. Pairing the logic with fast memory is what makes the transistor count useful at all.
Why does this matter for anyone building or using AI agents? Because when your agent feels sluggish, the delay usually is not the model “thinking harder.” It is data moving. Every token an agent generates requires reading through a large pile of model weights. The chip’s math units can chew through that work quickly. Getting the numbers in front of them is the slow part.
The invisible part underneath
There is an even less glamorous character in this story: the substrate. Think of it as an extremely precise circuit board that everything sits on, built from layers of a material called ABF. As designers pack more large logic dies alongside memory stacks on each accelerator, that base layer has to get bigger, flatter, and more intricate. It carries thousands of connections and has to stay dimensionally stable while hot silicon sits on top of it.
It is unglamorous in the way plumbing is unglamorous. Nobody tours a house to admire the pipes, right up until the pipes are the reason nothing works.
Follow the money and you find memory
The most interesting signal that memory is the real story comes from where investors are putting cash. In February 2026, Positron AI raised a $230 million Series B for what it describes as a memory-centric inference accelerator. Not a faster math engine. A chip organized around the movement of data.
A day earlier, Cerebras Systems closed a $1 billion Series H. Cerebras builds the Wafer Scale Engine 3, a processor that takes the whole “distance is the enemy” idea to its logical conclusion by refusing to cut the wafer into separate chips at all. Keeping compute and memory absurdly close together is the entire architectural bet.
What this means for the competitive picture
NVIDIA still holds an estimated 80 to 85 percent of data center AI accelerator revenue in 2026. That is a commanding position, and also a decline from roughly 92 percent in 2024. AMD has moved from around 2 percent to somewhere in the 5 to 7 percent range with parts like the MI300X+. Cerebras is pushing wafer-scale designs. Broadcom’s custom XPUs, built for specific customers rather than sold off the shelf, are gaining traction. Intel’s Gaudi 3, meanwhile, has underperformed.
Notice what the gainers have in common. They are not primarily competing on raw math throughput. They are competing on how memory and compute are arranged relative to each other, and on being built to order for a known workload. Custom XPUs win because a company that knows exactly what its models look like can size the memory system to match, instead of buying a general-purpose part and hoping.
The takeaway for non-engineers
You do not need to memorize part numbers. But if you are making decisions about AI agents, three ideas are worth carrying around:
- Memory bandwidth, not raw processing power, is usually what determines how fast an agent responds.
- Chip specs are describing an assembly of parts, not a single component, so headline transistor counts tell you less than you would think.
- Competition is shifting toward specialized designs, which should mean more choice and better prices for inference over time.
The chef gets the magazine cover. The countertops decide what dinner actually looks like.
🕒 Published: