Sign up and get 3 free requests with Start plan accessSign up →

Research finding

In the latest weekly window, 97.8% of eligible forecasts matched the other models' majority

Agreeing forecasts had a lower observed hit rate in this window; the disagreement group contained 108 forecasts.

If a model's forecast matches the consensus of the other models, is the forecast more reliable?

Updated 18 Nov 2026 · sources: Consensus Watch #15, Consensus Watch #14, Monthly Consensus Watch #3, Consensus Watch #13, Consensus Watch #12, Consensus Watch #11, Consensus Watch #10, Consensus Watch #9, Consensus Watch #8, Consensus Watch #7, Consensus Watch #6, Consensus Watch #5, Consensus Watch #4, Consensus Watch #3, Consensus Watch #2, Consensus Watch #1

All findingsThe benchmarkResearch reports

Answer

In Consensus Watch #15 (Nov 9-15, 2026 UTC), 97.8% of eligible forecasts matched the other models' leave-one-out majority, and agreeing calls scored 3.8pp lower than disagreeing calls (48.0% on n = 4,731 against 51.8% on n = 108). The 97.8% is the share of individual forecasts whose direction matched the leave-one-out majority of the other models; it is not the share of slots on which all 7 models agreed. The agreeing group carries a 95% Wilson interval of [46.6%, 49.5%] and the disagreeing group [42.5%, 61.1%]. A further 830 forecasts were thin or tied — fewer than three directional peers, or a peer split with no majority — and are excluded from both groups rather than forced into one.

Monthly Consensus Watch #3 pools the whole month: 98.7% peer-majority agreement on 19,271 scoreable calls, and agreeing calls scored 2.7pp higher than disagreeing calls (48.9% on n = 19,026 against 46.1% on n = 245). The weekly and the monthly readings do not have to point the same way, and in the published record they do not: the weekly lift reads worse for agreement, the monthly one better.

Across the 15 published weekly windows the sign of the lift changed 6 times, on disagreeing buckets that run from n = 6 to n = 125. The herd is large and steady; the difference between agreeing and disagreeing is not.

Evidence

Agreement against disagreement, leave-one-out

WindowPeer-majority agreementAgreeDisagreeLiftThin or tied
Consensus Watch #1597.8%48.0% (n = 4,731)51.8% (n = 108)-3.8pp830
Monthly Consensus Watch #398.7%48.9% (n = 19,026)46.1% (n = 245)+2.7pp3,308

Consensus Watch #15 · Nov 9-15, 2026 UTC · cutoff Mon 16 Nov 2026 16:00 UTC

Lift is the agreeing hit rate minus the disagreeing hit rate, in percentage points.

Hit rate by size of the agreeing majority — Consensus Watch #15

Peers on the same sidenHit rate
26943.5%
359248.6%
466051.1%
51,38049.4%
62,03046.1%

Consensus Watch #15 · Nov 9-15, 2026 UTC · cutoff Mon 16 Nov 2026 16:00 UTC

A cell needs at least three directional peers to be scored at all, so the thinnest majorities carry the smallest samples.

Hit rate by size of the agreeing majority — Monthly Consensus Watch #3

Peers on the same sidenHit rate
217441.9%
31,94050.0%
42,54544.7%
54,26648.4%
610,10150.0%

Monthly Consensus Watch #3 · Oct 1-31, 2026 UTC · cutoff Sun 1 Nov 2026 16:00 UTC

A cell needs at least three directional peers to be scored at all, so the thinnest majorities carry the smallest samples.

Across issues

Pooled lift, issue by issue

IssueWindowPeer-majority agreementAgreeDisagreeLift
Consensus Watch #1Aug 3-9, 2026 UTC99.4%40.4% (n = 3,308)63.2% (n = 19)-22.8pp
Consensus Watch #2Aug 10-16, 2026 UTC99.8%43.1% (n = 2,923)50.0% (n = 6)-6.9pp
Consensus Watch #3Aug 17-23, 2026 UTC97.1%54.8% (n = 4,232)37.6% (n = 125)+17.1pp
Consensus Watch #4Aug 24-30, 2026 UTC99.1%41.2% (n = 3,848)47.2% (n = 36)-6.0pp
Consensus Watch #5Aug 31-Sep 6, 2026 UTC99.3%41.6% (n = 3,725)69.2% (n = 26)-27.6pp
Consensus Watch #6Sep 7-13, 2026 UTC99.3%38.8% (n = 3,762)69.2% (n = 26)-30.4pp
Consensus Watch #7Sep 14-20, 2026 UTC98.9%50.6% (n = 4,825)38.5% (n = 52)+12.1pp
Consensus Watch #8Sep 21-27, 2026 UTC99.3%46.0% (n = 4,215)60.0% (n = 30)-14.0pp
Consensus Watch #9Sep 28-Oct 4, 2026 UTC99.5%51.0% (n = 3,682)55.6% (n = 18)-4.6pp
Consensus Watch #10Oct 5-11, 2026 UTC98.1%43.6% (n = 3,665)41.4% (n = 70)+2.2pp
Consensus Watch #11Oct 12-18, 2026 UTC98.8%46.3% (n = 4,276)44.4% (n = 54)+1.8pp
Consensus Watch #12Oct 19-25, 2026 UTC98.9%57.9% (n = 5,097)50.8% (n = 59)+7.0pp
Consensus Watch #13Oct 26-Nov 1, 2026 UTC98.7%40.6% (n = 4,289)49.1% (n = 55)-8.5pp
Consensus Watch #14Nov 2-8, 2026 UTC98.5%44.4% (n = 3,937)45.0% (n = 60)-0.6pp
Consensus Watch #15Nov 9-15, 2026 UTC97.8%48.0% (n = 4,731)51.8% (n = 108)-3.8pp

The sign of the lift changed 6 times across 15 weekly issues.

Caveats

The issue's own caveats, in its words:

Read with context. The disagree bucket is n=108 this week against 60 in issue #14 and 55 in issue #13; the two Wilson intervals overlap. This is a single week of dependent observations: not evidence about herding in general, and not a claim that the sign will hold next week.
Single week (Nov 9-15, 2026 UTC). Every finding is descriptive for this window only.
Observations are not independent: the same models watch overlapping symbol / FH / TF cells hour after hour. Wilson intervals here are descriptive, not inferential.
Correlational, not causal: all models see the same market data, so 'agreement' and 'hit' can rise together simply because a slot was easy to read. LOO removes self-match bias, not this confound.
Leave-one-out, as the issue defines it: leave-one-out strict majority, >=3 directional peers. A model is excluded from the consensus it is scored against, and a tied peer split is left out rather than forced either way.

Definitions

  • Method — leave-one-out strict majority, >=3 directional peers.
  • Agree — the forecast's direction matches the strict majority of the other model lines in the same cell. Disagree — its direction differs from the strict majority of the other models.
  • Margin — how many of the other lines stood on that majority side.

Agree, disagree and margin, as the issue defines them:

For every model M and every forecast it made, we rebuild that slot's consensus using only the OTHER active models in the same symbol / forecast-horizon (FH) / timeframe (TF) cell -- M's own call never counts toward its own consensus. M is scored agree if its side (long or short) matches the strict majority of those peers, and disagree if it sits alone against that majority; a cell needs at least 3 directional peers to count at all, and tied peer splits are excluded rather than forced either way. LOO peers are genuinely external to the model being scored, so the comparison below is not comparing a model to itself.

Reports and data

IssuePublishedFiles
Consensus Watch #1518 Nov 2026PDFMDJSON
Consensus Watch #1411 Nov 2026PDFMDJSON
Monthly Consensus Watch #35 Nov 2026PDFMDJSON
Consensus Watch #134 Nov 2026PDFMDJSON
Consensus Watch #1228 Oct 2026PDFMDJSON
Consensus Watch #1121 Oct 2026PDFMDJSON
Consensus Watch #1014 Oct 2026PDFMDJSON
Consensus Watch #97 Oct 2026PDFMDJSON
Consensus Watch #830 Sept 2026PDFMDJSON
Consensus Watch #722 Sept 2026PDFMDJSON
Consensus Watch #615 Sept 2026PDFMDJSON
Consensus Watch #58 Sept 2026PDFMDJSON
Consensus Watch #42 Sept 2026PDFMDJSON
Consensus Watch #326 Aug 2026PDFMDJSON
Consensus Watch #219 Aug 2026PDFMDJSON
Consensus Watch #110 Aug 2026PDFMDJSON

Dataset cardPublic APIDaily snapshots

Cite

MarketMania Research (2026). Research reports (weekly, monthly, quarterly), PDF/MD/JSON. https://marketmania.ai/research — dataset card: https://marketmania.ai/research/dataset

To cite one reading instead, name the issue it came from and the date it was published: an issue is frozen, so a citation to one is stable.

Research benchmark — not financial or investment advice; paper trading only.