4Paradigm and RiseUnion Open-Source ModelSphere for Heterogeneous LLM Serving

T2Covered by 1 source · first reported by China: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone at 16 Oct, 07:44 UTC

ModelSphere unifies cluster serving, KV caching and AutoTune under Apache 2.0, with claimed support for Ascend, Iluvatar, Moore Threads and Hygon DCU.

aidev

Sources

• T24Paradigm and RiseUnion Open-Source ModelSphere for Heterogeneous LLM ServingChina: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone16 Oct, 07:44