Wiring Beats Blending: Structure-Aware Compensation for Transformer Downscaling
arXiv:2608.02829v4 Announce Type: replace-cross Abstract: Model families are trained size by size. Can a pretrained large model instead be converted into a smaller sibling? We study the 1.4B->410M conversion in Pythia end to end. Representations align strongly across sizes (ridge R^2=0.84)…