Raw capture — raw/ai-superforecasters-scott-alexander.md
url: https://www.astralcodexten.com/p/the-ai-superforecasters-are-here title: The AI Superforecasters Are Here author: Scott Alexander date_fetched: 2026-07-29 date_published: 2026-07-02 site: Astral Codex Ten
Scott Alexander opens with an account of the annual prediction market conference, noting that prediction markets have grown from an obscure hobby into a multi-billion-dollar industry. He centers on the emergence of AI superforecasters — modified frontier models (like ChatGPT or Claude) with specialized "scaffolds" that guide them through extensive research processes.
Alexander tests FutureSearch, an AI forecasting company, on a question about a Silicon Valley initiative to end respiratory infections. He asks the probability that the rate of colds would be halved by 2040. Within two minutes, the AI deployed three subagents, read 16 websites, and after five minutes returned a 7% probability. The AI cited 212 sources and cost $8 in credits.
For comparison, another AI forecaster called Preseen gave 8.8%, and a human superforecaster gave 5–10%, depending on wording ambiguity.
Metaculus data shows AI approaching the wisdom-of-crowds community prediction but still trailing top professional superforecasters. However, forecasting-focused scaffolds add about nine months of progress beyond base models. The Metaculus Cup — a forecasting tournament — recently saw humans in the top two spots but an AI (Preseen) in third. Alexander describes this as effectively "a statistical dead heat."
Alexander draws a parallel to the folk legend John Henry, noting that top human forecasters are still holding out but likely not for long. Metaculus forecasters give "a 15% chance" a bot wins the summer 2026 cup and "a 95% chance" one wins before 2030.
He explains several reasons why AI might appear to outperform humans in finance specifically: data-heavy domains suit machines; AIs are faster and more diligent; and they can automate processes humans would find tedious.
Alexander notes the trendline of 0.9 Metaculus Elo points per month improvement. He highlights a prediction by Sayash Kapoor and Arvind Narayanan that AI would not surpass top human forecasters, calling it "a great test case" for whether human-level performance represents an "irreducible error" ceiling.
If the trend continues, he envisions transformed finance, strategic business decisions informed by AI forecasting, and potentially better policy decisions.
Alexander acknowledges a moment of cognitive dissonance: he asked about the probability of a US-China AI treaty and got 1–2.2% — a figure that challenged his own movement's priorities. He admits it was outside the distribution of things he trusts AIs on, noting "I still reject them the first time they really challenge my worldview."
Alexander suggests forecasting could solve the problem of AIs being barred from having opinions on controversial topics. Rather than saying "I don't have an opinion," a well-scaffolded AI could offer calibrated probabilities — a middle ground between fact and opinion. He envisions asking relationship advice and getting divorce-probability estimates.
Even as AI dominates forecasting, prediction markets retain value as canonical arbiters of truth — settling disputes between competing AI forecasts. AI reduces the cost of forecasting labor, allowing vastly more questions to be resolved. Alexander expects prediction markets to remain critical epistemic infrastructure, though he warns individual humans should not try to compete in them.
FutureSearch is publicly accessible, offering $20 in free credits. Preseen is in closed beta with a waitlist. Alexander suggests standard AIs have some forecasting capability (about 9–12 months behind specialized systems) with appropriate prompting.