Will AIs beat human experts in question-answering on the GPQA benchmark before January 1st, 2027?
One-OffAITechWetenschap3mnd
Manifold MarketsGeen KYCGeverifieerde afwikkelingsgegevensRedelijk gekalibreerdHuidige gemeenschapsvoorspelling
Ja 94.6%
Voorspellers
32
Vraagtype
binary
Methodologie
Play-money forecasting platform
Brontype
Voorspelling
Marktdata
Bijgewerkt 10 uur geleden
Verouderd18 jan 24, 00:11 UTC2 jan 27, 07:59 UTC
Trends
Gerelateerde Markten
Gerelateerde vragen voor context, geen vergelijking van gelijkwaardige contracten.
In what year will AI achieve a score of 95% or higher on the GPQA benchmark?
Manifold MarketsWill AI beat top human players at Civ6 (without cheating) by EOY 2026?
Manifold MarketsWill the US and Iran be at war on January 1st, 2027?
Manifold MarketsAI wins IMO gold medal in 2026?
PolymarketWhen will an AI lab solve another Millennium Prize Problem?
KalshiWhich country will be the next to send humans to the Moon?
KalshiRegels
From the abstract,
- We present GPQA, a challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry.
- We ensure that the questions are high-quality and extremely difficult: experts who have or are pursuing PhDs in the corresponding domains reach 65% accuracy (74% when discounting clear mistakes the experts identified in retrospect), while highly skilled non-expert validators only reach 34% accuracy, despite spending on average over 30 minutes with unrestricted access to the web (i.e., the questions are "Google-proof").
- The questions are also difficult for state-of-the-art AI systems, with our strongest GPT-4 based baseline achieving 39% accuracy.
- This question resolves to YES if a credible paper, blog post, or document of any kind indicates that at least some AI obtained a score of greater than 74.0% on the GPQA dataset before January 1st 2027, and NO otherwise.
- The result must be credible, and I will exclude results that appear to be the result of cheating: for example, results obtained by training on the test set.
Open markten
AIPOLYMARKETNext Claude Opus released on?
- September 22+41.9pt67.6%
- September 24+1.3pt12.4%
- September 23+0.7pt6.7%
+13 extra uitkomsten
- Totaal volume
- € 218,3K
- 24u volume
- € 178,7K
KALSHITime's Person of the Year for 2026
- Zohran Mamdani-1.0pt28%
- Mark Carney-1.0pt18%
- Donald Trump10%
+39 extra uitkomsten
- Totaal volume
- € 1,3 mln.
- 24u volume
KALSHIBest AI at the end of 2026?
- Claude-3.0pt70.3%
- ChatGPT+0.4pt13.9%
- Gemini-0.8pt7.9%
+5 extra uitkomsten
- Totaal volume
- € 10,2 mln.
POLYMARKETWill Anthropic’s valuation hit __ by December 31?
- ↑$1.25T+1.3pt100%
- ↑$1.1T+3.6pt100%
- ↑$1.0T+2.4pt100%
+10 extra uitkomsten
POLYMARKETWhich company has the best AI model end of September?
- Anthropic+0.1pt98%
- Google-0.3pt1%
- OpenAI+0.3pt1%
+18 extra uitkomsten
POLYMARKETAnthropic IPO by which date?
- December 31,+6.0pt79%
- December 15,+9.0pt74%
- November 30,+8.0pt57%
+7 extra uitkomsten
- Which company has best AI model end of 2026?
- Next Claude Opus released by which date?
- Best AI this week
- OpenAI announces another Millennium Prize solution by which date?
- When will Anthropic officially announce an IPO?
- Anthropic IPO Closing Market Cap
- Anthropic market share this week
- Best Chinese AI Company end of September?
- Stanley Cup Finals Winner
- U.S. enacts AI safety bill before 2027?
Veelgestelde vragen
- Wat zijn de huidige kansen voor Will AIs beat human experts in question-answering on the GPQA benchmark before January 1st, 2027??
- Manifold Markets prijst Yes op 95% per 21 september 2026.
- Wanneer wordt Will AIs beat human experts in question-answering on the GPQA benchmark before January 1st, 2027? afgewikkeld?
- Will AIs beat human experts in question-answering on the GPQA benchmark before January 1st, 2027? wordt naar verwachting op 2 januari 2027 afgewikkeld.
- Hoe wordt Will AIs beat human experts in question-answering on the GPQA benchmark before January 1st, 2027? afgewikkeld?
- Manifold Markets wikkelt deze markt als volgt af: From the abstract,