Moore Threads Open-Sources Megatron-MUSA-Titan for Trillion-Parameter MoE Training on Its GPUs

T2Covered by 1 source · first reported by China: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone at 29 Oct, 02:00 UTC

Moore Threads has open-sourced Megatron-MUSA-Titan, the training stack it uses for trillion-parameter MoE pretraining on 10,000-GPU clusters, with blockwise FP8 and support for slime and miles RL.

hardware

Sources

• T2Moore Threads Open-Sources Megatron-MUSA-Titan for Trillion-Parameter MoE Training on Its GPUsChina: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone29 Oct, 02:00