+20.4pp overconfidence gap — 62.8% stated confidence vs 42.4% realized◆33.6% — full unanimity was the weakest high-consensus tier◆0 / 1,890 high-confidence flips across identical-prompt repeats◆99.4% of directional calls ran with the herd◆Negative results get published too — that is the point◆One immutable snapshot frozen every day — the number you cite will not move◆

Independent research · live crypto markets

The continuous evaluation of frontier LLMs

Independent continuous evaluation infrastructure: a forward-only dataset of frozen multi-horizon forecasts — scored for calibration, stability and consensus, published as reproducible research.

Read the research→See the leaderboard

Research benchmark — not financial or investment advice. Paper trading only: no execution, no tailored recommendations.

Sandbox→top configs · week 33

#1 · 7d · week 33

+43.9%

+$439.1661 net over the 7d window

WR %·maxDD 5%·32 trades

fable+deepseek+opus+gpt · SOL,XRP · 4h/1h

Open this config in the sandbox →

#2 · 7d · fable+qwen+gemini+opus+43% · WR %#3 · 7d · gpt+opus+39.9% · WR %#1 · 30d · fable+deepseek+opus+qwen+86.8% · WR %#2 · 30d · gemini+gpt+deepseek+59.1% · WR %#3 · 30d · fable+deepseek+opus+qwen+58.1% · WR %

Best community configurations, re-simulated on the frozen report window · Config Watch #1 · paper only.

Indices→live · 15 Aug

+27

LLMs Crypto Market Outlook

BearishSidewaysBullish

HorizonSideSentiment1htf 1HBullish+31.14htf 4H + 1HBullish+26.51dtf 1D + 4HBearish−42.71wtf 1W + 1DSideways+10.61mtf 1M + 1WBearish−34.7

Weighted directional reading across all live model forecasts · headline: 4H horizon · more gauges: calibration, consensus.

Benchmarks→live · 7D · FH 1W

ModelTrades/ClosedNetEquityDeepSeek V4 Pro2 / 1+3.78$1,002.08Claude Opus 54 / 1+6.45$996.18Gemini 3.1 Pro4 / 1−1.03$991.11Claude Fable 56 / 3+0.60$982.37ChatGPT 5.6 Sol4 / 0+0.50$980.53Grok 4.54 / 1+4.73$977.33Qwen 3.8 Max5 / 3−2.26$973.16

5 of 7 models net-positive in this window · equity spread $28.92

Full leaderboard: 24H · 7D · 30D windows →

Official leaderboard · 7D window · FH 1W · TF 1D · $100 paper trades net of fees · equity includes earlier weeks

Join our TG channel→Follow us on X→

65,340forecasts recorded28,304settled trades7frontier models5assets9horizon/TF cells

Live counters · running since 15 Jul 2026 · one immutable snapshot frozen daily (UTC)

Latest research

Every report ships in PDF, Markdown and JSON — with methodology hash, source snapshot and BibTeX.

Weekly · Config WatchConfig Watch #1Every default configuration finished negative on both frozen windows, while search-tuned councils returned +43.9% (7d) and +86.8% (30d) in-sample — the out-of-sample test starts with issue #2.Weekly · CalibrationWeekly Calibration #1Frontier models were systematically overconfident this week: 62.8% stated confidence vs 42.4% realized directional accuracy.Weekly · Model WatchWeekly Model Watch #1The week's only trend reversal went uncalled: zero of 8 models caught BTC's +2.1% Monday turn.Weekly · ConsensusConsensus Watch #1More agreement did not mean more reliability: full peer unanimity was the weakest high-consensus tier this week.

What we test next: does the confidence-bucket inversion persist? · does full unanimity keep underperforming?

All reports →

Also available

Run AI-powered analysis on your own workflow

Beyond the benchmark, MarketMania Analyze structures live market data and sends it to the leading LLMs of your choice — with a combined summary, presets and scheduled delivery.