An Exploratory Ablation of a Small MLA--SSM Hybrid Language Model

T1Covered by 1 source · first reported by arXiv at 25 Sept, 04:00 UTC

arXiv:2609.29618v1 Announce Type: cross Abstract: We report an exploratory, single-seed ablation of TALH (Adaptive Latent Hybrid), a decoder-only language model with parallel Multi-head Latent Attention (MLA) and a custom recurrent state-space (SSM) branch. Five variants, spanning 117--217M…