Current community forecast
Yes 92.2%
Forecasters
116
Question type
binary
Methodology
Play-money forecasting platform
Source type
Forecast
Market data
Updated 1 hour ago
LaggingSep 16, 22, 8:43 PM UTCJan 1, 30, 8:00 AM UTC
Trends
Related Markets
Related questions for context, not a comparison of equivalent contracts.
In 2029, will any AI be able to take an arbitrary proof in the mathematical literature and translate it into a form suitable for symbolic verification? (Gary Marcus benchmark #5)
Manifold MarketsIn 2029, will any AI be able to work as a competent cook in an arbitrary kitchen? (Gary Marcus benchmark #3)
Manifold MarketsBy 2029, will any AI be able to read a novel and reliably answer questions about it? (Gary Marcus benchmark #2)
Manifold MarketsBy 2029 will any AI be able to watch a movie and accurately tell you what is going on? (Gary Marcus benchmark #1)
Manifold MarketsWhich company has best AI model end of 2026?
PolymarketBest AI at the end of 2026?
KalshiRules
The fourth question from this post: https://garymarcus.substack.com/p/dear-elon-musk-here-are-five-things
- The full text is: "In 2029, AI will not be able to reliably construct bug-free code of more than 10,000 lines from natural language specification or by interactions with a non-expert user. [Gluing together code from existing libraries doesn’t count.]"
- Judgment will be by me, not Gary Marcus.
- Ambiguous whether this means start or end of 2029, so I have set it for the end.
- For this question I am not using the exact text of the question, because I think "bug-free" is 1. silly 2. untestable.
- I will instead accept if it produces code of >=10k LOC with <= the number of bugs in an implementation by a human (many small bugs for some significant bugs will unfortunately be down to my subjective impression of whether it's "better")