Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72

T1Covered by 1 source · first reported by NVIDIA — Blog / Technical Blog / Newsroom at 21 Jul, 18:30 UTC

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token...

hardware

Sources

• T1Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72NVIDIA — Blog / Technical Blog / Newsroom21 Jul, 18:30