Question 1

What is MarketMania?

Accepted Answer

MarketMania is an independent research platform that benchmarks frontier LLMs on live crypto markets. Seven frontier models forecast the same assets every hour; valid forecasts are paper-traded under exchange-realistic rules, and the results feed weekly and monthly research reports, public leaderboards, a sandbox and an API. It is a research benchmark — not financial or investment advice. Paper trading only: no execution, no tailored recommendations.

Question 2

What exactly do you measure?

Accepted Answer

Every model receives the same disclosed payload (OHLCV, ten indicator sets, futures metrics and a news digest) and returns a structured forecast with a directional call, confidence, profit target and stop-loss. We score two things separately and never collapse them into one number: prediction quality (directional accuracy, coverage, Brier score, calibration gap) and trading performance (win rate, net and gross PnL, max drawdown, fees). Accurate and profitable are not the same thing.

Question 3

Is this financial advice? Can I trade from it?

Accepted Answer

No. MarketMania publishes a research benchmark. Nothing on the site or in its reports, leaderboards, sandbox, API or datasets is financial, investment, legal or tax advice, or a recommendation to buy or sell any asset. All trading shown is simulated (paper only) with mechanical rules — no orders are ever placed, and no output is tailored to any individual.

Question 4

Why are the models mostly losing money?

Accepted Answer

That is a finding, not a bug. We systematically measure where frontier models fail under real market conditions — and so far they do not hold risk discipline in choppy markets, which is consistent with the wider literature that LLMs are strong at retrieval but weak at prediction. We publish these negative results honestly rather than hiding them; that is what makes the benchmark useful to the labs whose models we measure.

Question 5

How do I know the results are not cherry-picked or overfit?

Accepted Answer

The benchmark is forward-only by construction: forecasts are recorded before outcomes are known, so nothing can be fitted after the fact. One immutable snapshot is frozen per day and is permanently addressable at /benchmarks/snapshots/<date> — the number you cite will not move under you. Every report carries a methodology version, source snapshot, lineage and a BibTeX citation.

Question 6

Which assets, horizons and models are covered?

Accepted Answer

Five assets — BTC, ETH, SOL, BNB and XRP — across five forecast horizons, refreshed hourly. The current field is seven frontier models (ChatGPT, Claude, Gemini, DeepSeek, Qwen and Grok), plus retired versions kept for lineage. Each model independently analyzes the same data, so the leaderboard is a like-for-like comparison. Exact model IDs and join dates are on the methodology page.

Question 7

Can I access the data and API?

Accepted Answer

Yes. Leaderboards, metadata, statistics and daily snapshots are available as JSON — for example the leaderboard endpoint and live stats — and every research report ships in PDF, Markdown and JSON. See Data for dataset access and Research for the reports.

Question 8

How is Research different from Benchmarks, Indices and the Sandbox?

Accepted Answer

They are interfaces to one continuously-running evaluation system. Benchmarks is the live leaderboard; Indices distils model behaviour into headline gauges (market outlook, calibration, consensus); the Sandbox lets you re-simulate a council configuration; and Research publishes the recurring reports and findings built on top of the growing dataset.

Question 9

How often is everything updated?

Accepted Answer

Forecasts are collected hourly and one immutable benchmark snapshot is frozen each day. Weekly reports (Calibration, Model Watch, Consensus) publish on Mondays; a monthly synthesis tests which findings replicate; and a quarterly flagship report summarises the season.

Question 10

What is Analyze, the paid product?

Accepted Answer

Analyze is our subscription SaaS: it collects and structures market data, then sends it to the AI models you select and shows their outputs side by side, with a combined summary. You can save presets and schedule runs with delivery to email or Telegram. It is a convenience layer around multi-LLM analysis — marketmania.ai does not perform analysis on its own and does not provide financial advice.

Question 11

How does scheduled analysis work?

Accepted Answer

You can save an analysis configuration as a preset and schedule it to run automatically at a chosen time. Results are delivered to your selected email and/or Telegram channel or private messages, so a preset can run and report on its own.

Question 12

How are Analyze requests counted?

Accepted Answer

Each request is tied to one AI model run. If an analysis is launched with 2 selected models, 2 requests are used; with 3 models, 3 requests. The number of tickers, indicators, futures metrics and other settings does not change the number of requests deducted.

Frequently Asked Questions