Self-Play Pretraining with Zero Data
arXiv:2609.30063v1 Announce Type: new Abstract: Advances in language modeling have been driven by scaling pretraining on ever more data. Yet, the training data is still largely curated on the model's behalf. A more general approach to pretraining would let the model learn to generate the data most…