Canopy: Exploiting Piecewise Smooth Tree Priors for Multi-Fidelity Bandits
arXiv:2609.30017v1 Announce Type: cross Abstract: Many LLM inference problems, including model routing, prefix-cache management, prompt trimming, and test-time search, can be viewed as optimization over a tree. This structure arises naturally from autoregressive generation: every prefix defines a…