If you peek inside today’s most advanced AI systems, you won’t find a single monolithic brain working overtime. Instead, you’ll find something far closer to how humans actually think: many specialists, only a few of them active at any given moment.
This design philosophy, known as Mixture of Experts (MoE), has quietly become the backbone of frontier AI. And only now, with the arrival of NVIDIA’s Blackwell GB200 NVL72 systems, is the industry finally able to run these models at full speed, at a fraction of the cost.
The result is a major shift in the economics of AI: models that run ten times faster while driving token costs down by roughly 90%.
From brute force to selective intelligence
For years, AI progress followed a simple rule: bigger was better. Models grew denser, meaning every question activated every part of the network, hundreds of billions of parameters firing simultaneously to generate each word. The intelligence gains were real, but so were the costs, both in power and money.
MoE takes a different route. Instead of using the whole model for every task, it splits intelligence into specialized components or “experts.” A routing mechanism decides which experts are relevant for each token and activates only those.
The analogy is straightforward: your brain doesn’t engage your entire neural system to solve a math problem or recognize a face. MoE models work the same way. They carry the knowledge of massive models but use only a small slice of it at any moment.
This selective approach delivers higher efficiency without sacrificing capability and the industry has taken notice.
According to the independent Artificial Analysis leaderboard, all of the top 10 most capable open-source models now rely on MoE architectures. That list includes DeepSeek-R1, Kimi K2 Thinking, OpenAI’s gpt-oss-120B, and Mistral Large 3.
More broadly, over 60% of open-source AI models released this year use MoE designs. Since early 2023, this shift has supported an estimated 70-fold jump in model intelligence, pushing AI systems closer to human-like reasoning efficiency.
As Guillaume Lample, cofounder and chief scientist at Mistral AI, has noted, MoE is what makes it possible to scale intelligence while keeping energy and compute demands in check, a necessity as AI moves from labs into real-world products.
Why MoE is hard to run in practice
Despite its elegance, MoE has a serious weakness: scale.
These models are too large to live on a single GPU. Their experts must be spread across many GPUs, a setup known as expert parallelism. On earlier hardware platforms, including NVIDIA’s H200, this created two major problems.
First, memory strain. Each GPU had to constantly load expert parameters in and out of high-bandwidth memory, creating bottlenecks.
Second, communication delays. Once experts were spread beyond a small cluster of GPUs, they had to exchange information over slower interconnects. That latency erased many of the theoretical gains of MoE.
In short, the software vision outpaced the hardware reality.
The rack-scale shift that changed everything
The Blackwell GB200 NVL72 marks a fundamental departure from traditional GPU servers. Instead of treating GPUs as loosely connected units, it links 72 Blackwell GPUs into a single system using NVLink Switch, creating what functions as one massive processor.
The system delivers 1.4 exaflops of AI performance and 30 terabytes of shared memory, with every GPU able to communicate with every other at ultra-high speed, up to 130 terabytes per second.
This matters enormously for MoE. Experts can now be spread thinly across dozens of GPUs, reducing memory pressure on each chip while allowing near-instant communication between them. In some cases, the NVLink Switch itself even helps with the math required to combine expert outputs.
The result is that MoE models can finally scale the way they were meant to.
When deployed on GB200 NVL72, leading MoE models show dramatic gains. Kimi K2 Thinking, currently ranked as the most intelligent open-source model, runs ten times faster than it does on H200-based systems. The same leap applies to DeepSeek-R1 and Mistral Large 3.
Speed matters but cost matters more.
Because the system processes roughly ten times as many tokens using comparable time and power, the cost per token drops by more than an order of magnitude. SemiAnalysis data shows that for models like DeepSeek-R1, the cost to generate a million tokens on Blackwell is over ten times lower than on H200 at the same response latency.
Hardware alone didn’t solve the problem. NVIDIA’s Dynamo framework helps divide workloads intelligently, assigning different GPUs to the “prefill” and “decode” stages of inference. Meanwhile, formats like NVFP4 preserve accuracy while improving efficiency.
Open-source tools, including TensorRT-LLM, SGLang, and vLLM, have matured alongside the hardware, enabling large-scale MoE deployments that were previously impractical.
This full-stack approach is why cloud providers such as AWS, Google Cloud, Microsoft Azure, Oracle, and specialized AI clouds like CoreWeave and Together AI are now rolling out GB200 NVL72 systems.
Modern multimodal models already contain separate components for language, vision, and audio. Agentic AI systems go further, coordinating multiple specialized agents for planning, reasoning, searching, and tool use.
In all cases, the pattern is the same: route tasks to the right specialists, then combine their outputs efficiently.
At scale, this enables a shared pool of digital experts serving many applications and users, without duplicating massive models each time. That efficiency is what makes widespread, sustainable AI deployment possible.
The industry has reached a consensus, even if it hasn’t said so explicitly: the future of AI isn’t about lighting up every neuron every time. It’s about using intelligence selectively, backed by hardware that can keep up.
Mixture of Experts made that vision possible. Blackwell GB200 NVL72 made it practical.
And with NVIDIA’s roadmap extending toward the Vera Rubin architecture, this shift toward scalable, efficient intelligence looks more like the foundation of AI’s next decade.
