Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
arXiv:2609.29845v1 Announce Type: cross Abstract: While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams are linearly combined, the model outputs a superposition of the…