End of pre-training era for language models: Will an LM fine-tune for more FLOPs than it is pre-trained for, before 2026
KIOne-Off2Mon
Manifold MarketsKein KYCVerifizierte AuflösungsdatenOrdentlich kalibriertAktuelle Community-Prognose
Nein 63.2%
Nein 63.2%Prognostiker
24
Fragetyp
binary
Methodik
Play-money forecasting platform
Quellentyp
Prognose
Marktdaten
Aktualisiert vor 11 Stunden
Veraltet5. Mai 23, 23:06 UTC2. Jan. 27, 04:59 UTC
Trends
Regeln
The LM must have 'frontier' performance (having PILE/or similar perplexity above one year prior's SotA).
- The LM must have been trained after 2022.
- If it's unclear whether this has happened, I will give this a year to resolve.
- If it remains plausibly unclear the market will resolve N/A. Fine-tuning includes all RL training.
- Training on synthetic data, or additional supervised learning (which is deliberately trained on after training on a PILE-like generic dataset) counts as fine-tuning.
