Ant Group and vLLM Open-Source Humming, Quantized Inference Kernels With Zhenwu 810E Support

T2Covered by 1 source · first reported by China: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone at 21 Oct, 06:49 UTC

Ant Group and the vLLM community open-sourced Humming, a JIT GEMM kernel library for quantized inference. It covers INT1-8 and FP4-FP8 mixes and fully supports T-Head's Zhenwu 810E.

aidev

Sources

• T2Ant Group and vLLM Open-Source Humming, Quantized Inference Kernels With Zhenwu 810E SupportChina: SCMP Tech / TechNode / Pandaily / KrASIA / Sixth Tone21 Oct, 06:49