METASTONE Launches TokenFleet to Schedule LLM Inference Across GPU Clusters and Data Centers

T2Covered by 1 source · first reported by China: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone at 21 Oct, 01:55 UTC

Compute operator METASTONE unveiled TokenFleet, a scheduling framework for LLM inference clusters. Its slides show P95 time-to-first-token down 32.4% and on-target output throughput up about 29.4%.

aiclouddevhardware

Sources

• T2METASTONE Launches TokenFleet to Schedule LLM Inference Across GPU Clusters and Data CentersChina: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone21 Oct, 01:55