Sign up and get 3 free requests with Start plan accessSign up →
+20.4pp overconfidence gap — 62.8% stated confidence vs 42.4% realized◆33.6% — full unanimity was the weakest high-consensus tier◆0 / 1,890 high-confidence flips across identical-prompt repeats◆99.4% of directional calls ran with the herd◆Negative results get published too — that is the point◆One immutable snapshot frozen every day — the number you cite will not move◆

Independent research · live crypto markets

The continuous evaluation of frontier LLMs

Independent continuous evaluation infrastructure: a forward-only dataset of frozen multi-horizon forecasts — scored for calibration, stability and consensus, published as reproducible research.

Research benchmark — not financial or investment advice. Paper trading only: no execution, no tailored recommendations.

Indices→live · 22 Nov
+53
LLMs Crypto Market Outlook
BearishSidewaysBullish
HorizonSideSentiment1htf 1HBullish+83.44htf 4H + 1HBullish+52.51dtf 1D + 4HBearish−26.31wtf 1W + 1DBearish−82.21mtf 1M + 1WBearish−82.0

Weighted directional reading across all live model forecasts · headline: 4H horizon · more gauges: calibration, consensus.

Benchmarks→live · 7D · FH 1W
ModelTrades/ClosedNetEquityGPT-6.1 Sol41 / 29+202.70$1,202.70Gemini 3.1 Pro30 / 18+139.36$1,139.36DeepSeek V4 Pro31 / 27+97.30$1,097.30Grok 4.737 / 28+53.96$1,053.96Qwen 3.8 Max23 / 18+46.83$1,046.83Claude Fable 525 / 17+43.52$1,043.52Claude Opus 5.510 / 7+10.91$1,010.91
7 of 7 models net-positive in this window · equity spread $191.79
Full leaderboard: 24H · 7D · 30D windows →

Official leaderboard · 7D window · FH 1W · TF 1D · $100 paper trades net of fees · equity includes earlier weeks

210,630forecasts recorded107,346settled trades7frontier models5assets9horizon/TF cells

Live counters · running since 15 Jul 2026 · one immutable snapshot frozen daily (UTC)

Latest research

Every report ships in PDF, Markdown and JSON — with methodology hash, source snapshot and BibTeX.

What we test next: does the confidence-bucket inversion persist? · does full unanimity keep underperforming?

Also available

Run AI-powered analysis on your own workflow

Beyond the benchmark, MarketMania Analyze structures live market data and sends it to the leading LLMs of your choice — with a combined summary, presets and scheduled delivery.

Tech Indicators and metricsTech Indicators and metrics

Tech Indicators and metrics

Charts do not fit into a chat — payloads do. Every run packs price action, 10+ indicator sets and futures metrics into one model-ready brief

Multi-Exchange DataMulti-Exchange Data

Multi-Exchange Data

Pick any of 100+ spot and futures pairs — the market picture assembles itself in seconds, fresh news digest included

AI-Powered AnalysisAI-Powered Analysis

AI-Powered Analysis

One question, six frontier minds: ChatGPT, Claude, Gemini, DeepSeek, Qwen and Grok answer side by side, with a single combined verdict on top

Frequently Asked Questions

See all questions →