CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference

T1Covered by 1 source · first reported by arXiv at 24 Sept, 04:00 UTC

arXiv:2609.26300v1 Announce Type: cross Abstract: Despite their strong performance, large language models (LLMs) are bottlenecked by KV cache memory traffic during long-context inference. Sparse attention is widely used to accelerate LLM inference by computing exact attention over a selected subset…

aiscience

Sources

• T1CompKV: Compensation-Aware KV Selection for Long-Context LLM InferencearXiv24 Sept, 04:00