Cerebras Runs 3T Kimi Model on 59 Wafers
Cerebras shows how its wafer-scale chips run the 3-trillion-parameter Kimi K3 model across 59 wafers, keeping weights on chip for 1,000-2,000 tokens per second.
Chipmaker Cerebras has shown how a single giant AI model can run across 59 of its wafer-scale chips. The worked example uses Kimi K3, a roughly 3-trillion-parameter open model from Moonshot AI. The trick is simple to state: keep the model's weights on the chips themselves, and pass only small intermediate results between them.
Why one model needs 59 wafers
A single Cerebras WSE-3T wafer holds 44 GB of on-chip memory. A 3-trillion-parameter model needs far more than that for its weights alone, so Cerebras spreads the model across many wafers and links them into a pipeline. At 4-bit weight quantization, each of Kimi K3's 93 layers takes about 16 GB. The layer type decides how many fit on one wafer. Dense layers need extra room for a key-value cache, so each dense layer gets a whole wafer. Linear-attention layers keep only a small fixed state per request, so two fit on one wafer. Kimi K3 has 69 linear layers and 24 dense layers. That works out to 35 wafers for the linear layers and 24 for the dense layers, giving 59 wafers in total.
Weights stay put, only activations move
Cerebras runs models as a pipeline. After its compiler maps layers across the wafers, tokens flow from one wafer to the next. Because whole layers fit on the chips, the weights never have to move. Only activations cross between wafers, carried by RDMA over Ethernet at about 2 microseconds per hop. The payoff, according to Cerebras, is output speed of 1,000 to 2,000-plus tokens per second for multi-trillion-parameter models. Tokens are the small pieces of text a model reads and writes. Kimi K3 itself is reported to score 88.3 percent on Terminal-Bench 2.1 and 91.2 percent on BrowseComp.
What is still unproven
The post is a deployment recipe, not a measured benchmark result. Cerebras has not shown an independently verified test tying this exact Kimi K3 layout to the advertised speed, and it has not published full benchmark conditions or measured power use for the setup. The useful advance is concrete: a clear answer to how much hardware a model of this size actually demands. Still, the math points to a real shift. As open models grow past a trillion parameters, running them well is becoming a memory-placement problem as much as a chip-speed problem. Cerebras is betting that keeping weights on chip, and moving as little data as possible, is the way to solve it.
Sources
- Cerebras - https://www.cerebras.ai/blog/how-we-deploy-trillion-parameter-frontier-models-on-cerebras - Daily Inference - https://dailyinference.news/articles/2026-10-25-59-wafers-to-hold-one-trillion-parameter-model-cerebrass-math-says/ - LavX News - https://news.lavx.hu/article/how-cerebras-deploys-trillion-parameter-ai-models-on-wafer-scale-chips