MarketMania is an independent research platform that benchmarks frontier LLMs on live crypto markets. Seven frontier models forecast the same assets every hour; we paper-trade the valid calls under exchange-realistic rules, freeze an immutable snapshot every day, and turn the results into recurring research — leaderboards, indices, a sandbox and a public API.
Research benchmark — not financial or investment advice. Paper trading only: no execution, no tailored recommendations. Model forecasts are experimental and frequently wrong; past simulated results do not indicate future results.
Every model receives the same disclosed payload — OHLCV, ten indicator sets, futures metrics and a news digest — and returns a structured forecast with a directional call, confidence, profit target and stop-loss. We score two things separately and never collapse them into a single “magic score”: prediction quality (directional accuracy, coverage, Brier score, calibration gap) and trading performance (win rate, net and gross PnL, max drawdown, fees). Accurate and profitable are not the same thing, and we publish negative results — overconfidence, herd failures, losing weeks — honestly.
Crypto is a rare-quality evaluation environment for language models: it trades 24/7, so the benchmark collects the maximum number of fresh, forward-looking tasks per day; the ground truth is objective and arrives on a fixed clock; and the cost of a wrong call is measured in dollars, not opinion. Because every forecast is recorded before its outcome is known, the benchmark is forward-only by construction — nothing can be fitted after the fact, and the daily frozen snapshots prove it.
MarketMania is a small, self-funded project, pre-incorporation. We benchmark the same labs whose models we would ask for API credits, so we hold a simple rule: credits are not placement. Any vendor credit we receive will be logged publicly, per lab, and no result is ever pre-shown or adjusted for a vendor. Every report is reproducible — it carries a methodology version and hash, the exact source snapshot, model lineage and a BibTeX citation, and daily snapshots are permanently addressable so a number you cite will not move under you.
Alongside the research benchmark, MarketMania offers a subscription product, Analyze. It collects and structures live market data and sends it to the AI models you select, showing their outputs side by side with a combined summary; you can save presets and schedule runs with delivery to email or Telegram. Analyze is a convenience layer around multi-LLM analysis — marketmania.ai does not perform analysis on its own and does not provide financial, investment, legal or tax advice, guarantee accuracy or outcomes, or replace independent research and personal risk management. Use responsibly.
Contact: contact@marketmania.ai
Support: support@marketmania.ai