In short
In short: Mixture-of-Experts (MoE) is a model design where the model is split into many specialized parts, and for every word of the answer only a few of them switch on. So a model with an enormous total size works with a small part of itself at any moment - faster and cheaper, without losing the knowledge of the whole.
A hospital does not have its entire medical staff examine you. At the front desk they route you: this one is for the cardiologist, this for the eye doctor, this for the lab. The other doctors are seeing other patients meanwhile. An MoE model is a hospital for words: a small dispatcher decides which specialist takes exactly this piece of the sentence, while the rest stay silent and burn no power.
The clever part is that the specialization is not assigned by people - it self-organizes during training. Nobody says “you will handle chemistry.” Some parts simply start firing on code, others on numbers, others on rare languages. There are no labels; the division works.
A big model that thinks with a small part of itself: that is the deal that made the giants affordable.
Why this matters if you don't build models
Because it explains the numbers in the news. When you read that a model has an enormous parameter count yet runs fast and cheap, MoE is almost always underneath: the total size is the warehouse, the active part is the bill. And when you compare models by size, the comparison is no longer fair if one is dense and the other a mixture of experts.
A fresh example is Reflection's Beam, released on October 16, 2026: a mixture of experts with 501 billion parameters, of which only 23 billion are active for each word (as presented on Hugging Face). The warehouse is 501, the bill is 23. The quality numbers Reflection publishes for it are its own and do not compare it with other models.
Our read: MoE is one more proof that progress in AI does not come only from “more”. It comes from more, arranged more cleverly. The warehouse grows, but you only pay for what leaves it.
Primary sources
Hugging Face: Reflection's Beam (501B parameters, 23B active per token), 16.10.2026Reflection: Beam, the first open-weight model, 501B MoEHugging Face: Mixture of Experts ExplainedShazeer et al.: Outrageously Large Neural Networks - the Sparsely-Gated MoE layer (arXiv, 2017)The visual is generated code-art, no third-party images.