+20.4pp overconfidence gap — 62.8% stated confidence vs 42.4% realized◆33.6% — full unanimity was the weakest high-consensus tier◆0 / 1,890 high-confidence flips across identical-prompt repeats◆99.4% of directional calls ran with the herd◆Negative results get published too — that is the point◆One immutable snapshot frozen every day — the number you cite will not move◆

Independent research · live crypto markets

The continuous evaluation of frontier LLMs

Independent continuous evaluation infrastructure: a forward-only dataset of frozen multi-horizon forecasts — scored for calibration, stability and consensus, published as reproducible research.

Read the research→See the leaderboard

Research benchmark — not financial or investment advice. Paper trading only: no execution, no tailored recommendations.

Sandbox→top configs · week 40

#1 · 7d · week 40

+58.3%

+$583.3534 net over the 7d window

WR 0.7317%·maxDD 22.7%·41 trades

gpt+gemini+qwen · SOL,XRP · 4h/4h,1h

Open this config in the sandbox →

#2 · 7d · gpt+gemini+qwen+grok+57.1% · WR 0.6316%#1 · 30d · fable+opus+gemini+qwen+101.9% · WR 0.7253%#2 · 30d · fable+opus+gemini+qwen+94.3% · WR 0.697%

Best community configurations, re-simulated on the frozen report window · Config Watch #8 · paper only.

Indices→live · 1 Oct

+37

LLMs Crypto Market Outlook

BearishSidewaysBullish

HorizonSideSentiment1htf 1HSideways+6.24htf 4H + 1HBullish+36.51dtf 1D + 4HBullish+27.71wtf 1W + 1DBullish+81.21mtf 1M + 1WBullish+86.2

Weighted directional reading across all live model forecasts · headline: 4H horizon · more gauges: calibration, consensus.

Benchmarks→live · 7D · FH 1W

ModelTrades/ClosedNetEquityGemini 3.1 Pro50 / 20+7.32$1,007.32Grok 4.640 / 19+1.14$1,001.14ChatGPT 5.6 Sol55 / 22−3.49$996.51Qwen 3.8 Max46 / 18−8.11$991.89Claude Fable 537 / 16−14.29$985.71Claude Opus 536 / 14−32.03$967.97DeepSeek V4 Pro44 / 18−40.16$959.84

2 of 7 models net-positive in this window · equity spread $47.48

Full leaderboard: 24H · 7D · 30D windows →

Official leaderboard · 7D window · FH 1W · TF 1D · $100 paper trades net of fees · equity includes earlier weeks

Join our TG channel→Follow us on X→

134,439forecasts recorded63,662settled trades7frontier models5assets9horizon/TF cells

Live counters · running since 15 Jul 2026 · one immutable snapshot frozen daily (UTC)

Latest research

Every report ships in PDF, Markdown and JSON — with methodology hash, source snapshot and BibTeX.

Weekly · Config WatchConfig Watch #8The ceiling of the fresh search moved from +$1,467.8 to +$583.4 on the week and 24 of the 48 cards this series has published are net-positive on that same week, and under the standing two-window rule 10 of the 48 Survive: 16 clear the month, 24 clear the week.Weekly · Model WatchWeekly Model Watch #8Field accuracy moved 50.7% -> 46.8% (-3.9pp) with 0 of 7 models improving; gemini-3.1-pro led the week at 51.4% (-1.5pp w/w) and no weekly title was awarded, for the eighth issue running.Weekly · CalibrationWeekly Calibration #8The overconfidence gap read +16.1pp against +12.1pp in issue #7, with 0 of 7 model lines narrowing and field Brier moving 0.2669 -> 0.2751; trading, kept separate, finished 0 of 7 lines net-positive.Weekly · ConsensusConsensus Watch #8Herding ran at 99.3% of leave-one-out scoreable calls and the pooled lift came in at -14.0pp: agree-hit 46.0% against 60.0% on disagreement (disagree bucket n=30), against a 46.8% field base.

What we test next: does the confidence-bucket inversion persist? · does full unanimity keep underperforming?

All reports →

Also available

Run AI-powered analysis on your own workflow

Beyond the benchmark, MarketMania Analyze structures live market data and sends it to the leading LLMs of your choice — with a combined summary, presets and scheduled delivery.