Guide
Why a 30B MoE model runs faster than an 8B dense one
Mixture-of-experts models need the memory of a large model and the bandwidth of a small one. That trade is a gift for some hardware and a trap for others.
A dense model reads every weight to produce every token. A mixture-of-experts model routes each token to a handful of expert sub-networks and ignores the rest. Qwen3 30B-A3B has 30.5 billion parameters and touches 3.3 billion of them per token.
Two different budgets
Capacity is set by the total. All the experts have to be resident, so a 30B MoE model occupies the memory of a 30B dense model — around 18.6 GB at Q4_K_M.
Speed is set by the active count. Generation is bound by how many bytes must be read per token, and reading two gigabytes is far quicker than reading eighteen. The result is a model with the memory footprint of a 30B and the throughput of something closer to a 3B.
Which hardware this suits
Anything with plenty of memory and unremarkable bandwidth. A DGX Spark has 128 GB at 273 GB/s: a dense 70B model crawls on it, while a 30B MoE runs at a comfortable conversational speed. The same applies to Apple Silicon with large unified memory, and to Strix Halo systems.
Conversely, on a card with high bandwidth and little memory — an RTX 5080, say — MoE models are a poor fit. You pay full price in capacity for a model you cannot load and would not have needed the sparsity for.
The catch
Sparse reads are less efficient than sequential ones. Routing scatters memory access across experts, so real throughput lands a little below what the active-parameter count alone predicts — the estimates here apply roughly a 14% penalty for this. Batched serving suffers more, because different sequences in a batch route to different experts and the savings partly evaporate.
Quality is the other consideration. A 30B-A3B model is stronger than a 3B dense model by a wide margin, but it is not equal to a 30B dense model. You are buying speed with parameters that sit idle.
Offloading experts is a real strategy
Because only a few experts are needed per token, some runtimes keep the attention layers and shared weights in VRAM and stream expert weights from system memory. It works better than offloading a dense model does, and it is how people run gpt-oss-120b and larger MoE models on a single consumer card at usable speed. Expect a substantial slowdown, but not the collapse that dense offloading causes.