We simulated 200,000 backtests per sample size for a strategy with no edge at all, and for one with a real but modest edge. Short answer: at 20–50 trades a 60% win rate tells you almost nothing.

Published 25 Oct 2026 · Code and output saved with fixed random seed 20261025 · Simulation, not market data

The short version

A strategy with zero edge shows a 60%+ win rate in 25.2% of 20-trade backtests, 10.1% of 50-trade backtests, 2.8% at 100 trades, and 0.03% at 300. And the reverse problem is just as big: a strategy that truly wins 55% of the time shows 50% or worse in 40.9% of 20-trade samples and still 18.3% of 100-trade samples.

The experiment

Every simulated trade either wins 1R or loses 1R (same size win and loss), so the win rate is the whole story. We used two strategies:

• Coin flip: true win rate 50% — no edge whatsoever.
• Real edge: true win rate 55% — an expected +0.10R per trade before costs, which would be a genuinely good strategy.

For each sample size (20, 50, 100, 300 trades) we ran 200,000 backtests of each strategy and counted how often the result looked misleading. We also computed the exact binomial probabilities; they match the simulation to within 0.1 percentage points.

(Bar chart on the live page: how often a small sample fools you — no-edge strategy shows ≥60% wins vs real 55% edge shows ≤50% wins, across 20 / 50 / 100 / 300 trades. See the canonical article.)

"Best of 10/50" = you tried 10 or 50 variations (different settings, filters, timeframes) of a strategy that has no edge, and kept the best one. p-value: one-sided exact binomial test against a 50% win rate.

What this means in plain English

• Under about 100 trades, a 60% win rate is not evidence. At 20 trades, one in four no-edge strategies gets there by luck. At 50 trades it is still one in ten.
• Trying many versions makes it much worse. Tweak a no-edge idea 10 times on 50 trades each and the best version shows 60%+ wins 65.6% of the time. Most "optimised" backtests are exactly this.
• Small samples also hide real edges. A genuine 55% strategy looks like a loser or break-even in 28.3% of 50-trade samples. Abandoning a strategy after 30 bad trades can be as wrong as trusting one after 30 good ones.
• How many trades is enough? To have an 80% chance of telling a real edge from luck (one-sided test at the 5% level), you need about 158 trades if the true win rate is 60%, 620 if it is 55%, and 3,886 if it is 52%. The smaller the edge, the more trades you need.

Checklist before you trust a backtest

1. Count the trades. Fewer than ~100: treat any result as a hypothesis, not a finding.
2. Count how many variants you tried. Every extra parameter tweak raises the odds that the best one is luck.
3. Test on data you did not use to build the rules (out-of-sample), and with real spreads and slippage.
4. Look at expectancy (average R per trade), not just win rate: a 60% win rate with small wins and big losses can still lose money.

Assumptions and limitations

• Trades are independent and every win/loss is exactly ±1R. Real strategies have variable wins and losses, and losing streaks can cluster, which usually makes small samples less reliable than shown here.
• No trading costs. Costs shift every result down; a 55% win rate at ±1R can turn into a loser once spread and slippage are included.
• For strategies with uneven wins and losses, the right check is a t-test or bootstrap on per-trade R (or returns), not a win-rate test. Our example checkup does exactly that.
• "Trades needed" uses an exact binomial test (one-sided, 5% level, 80% power) — a standard but not unique choice.

Method: numpy random binomial draws (200,000 per case, seed 20261025) plus exact scipy binomial probabilities. Script and full output are kept on file and available on request.

Want this check run on your own strategy?

Send us your rules or your MT4/MT5 trade history. We reply with a plain-English report: sample size vs luck, out-of-sample, what is left after costs, and the worst case. Your strategy stays private unless you opt in.

• Get a free strategy checkup
• See a sample report

Research and education only — not investment advice and not a trading signal. Nothing here is a recommendation to use any strategy; any rules discussed were tested to show how backtests mislead, not to suggest trades. Past results do not guarantee future returns. Trading leveraged products such as gold CFDs can lose you more than you deposit. Full disclaimer · Privacy · Terms · X @ForwardProof_AI

Real hourly XAUUSD prices from 2015 to 2026, real bid/ask spreads, a clean in-sample / out-of-sample split. Before costs, 40 rules looked statistically "significant". After costs, 4 did. None beat simply holding gold.

Published 25 Oct 2026 · Data: Dukascopy XAU/USD hourly bid and ask, 2015-01-01 to 2026-10-16 (69,827 bars) · Rules drawn at random with a fixed seed

The short version

• Before costs, half the rules made money in-sample (495 of 1,000) — about what chance predicts. After real spreads and slippage, 214 did.
• 40 rules passed a standard significance test (t ≥ 2) before costs — if no rule had any edge and the tests were independent, you would expect about 23 by luck. Most were very active rules (median 3,611 position changes in-sample). After costs only 15 of those 40 were still profitable in-sample; their median went from +36.5% to −22.9%.
• After costs, just 4 of 1,000 rules were significant in-sample — all RSI reversal rules with extreme thresholds that were in the market only about 0.5% of the time or less, and made a few percent in total. Out-of-sample, 2 of the 4 stayed positive and 0 stayed significant.
• Out-of-sample (2022 to Oct 2026), 350 of 1,000 rules made money after costs; the median rule lost 5.6%. Not one beat buying and holding gold (+90.5% measured the same way; +127.9% compounded).
• No rule was significant after costs in both periods.

The chart

(On the live page: each dot is one rule — its after-cost result in 2015–2021 on the horizontal axis against 2022–2026 on the vertical. Red dots are the busiest rules. See the canonical article for the interactive/full chart.)

The surprise: in-sample rank did carry over — for the wrong reasons

The usual overfitting story is "the best backtest collapses out-of-sample". That is not quite what happened here. Ranked by in-sample result, the rules kept much the same order out-of-sample (rank correlation 0.81). Median out-of-sample result by in-sample fifth, worst to best: −60.6%, −22.1%, −9.1%, 0.0%, +43.1%. The 20 best in-sample rules had a median of +34.5% in-sample and +59.3% out-of-sample; 18 of 20 stayed positive.

But two boring things explain most of that persistence, and neither is a clever signal:

• Costs. Rules that trade a lot lose to the spread in every period. The median rule gave up 19.9 percentage points to costs in-sample; the busiest families gave up far more (table below). A rule that loses to costs in 2015–2021 keeps losing to costs in 2022–2026.
• Being long a rising market. Gold rose in both periods (+54.4% in 2015–2021 and +127.9% in 2022–2026, compounded). Out-of-sample, the rank correlation between a rule's net long exposure and its result was 0.70. The best rules were mostly slow trend-followers (moving-average crossover and breakout rules), many long-only, that simply spent more time long gold. Of the 17 rules that were "significant" out-of-sample, 15 were long-only.

Even the single best in-sample rule — a 43/118-hour moving-average crossover, long and short — went +55.3% in-sample (403 position changes, t = 1.52, not significant) and +47.2% out-of-sample, with maximum drawdowns of 29.7 and 26.8 percentage points. Holding gold did better (+90.5% out-of-sample, same arithmetic). Some of it is more than exposure: of the 20 rules that beat an exposure-matched buy-and-hold by the most in 2015–2021 (mostly slow trend rules), 15 also beat it in 2022–2026. That is consistent with gold trending over this decade — but no rule was significant after costs in both periods, and this is one market over one decade.

By rule family

Cost drag = before-cost result minus after-cost result, in percentage points of summed return.

What to take from this

• Most "great" backtests are made before costs. Most of the rules that looked significant on a frictionless test were busy rules that costs then wiped out.
• If you test many rules, some will pass a significance test by luck. 1,000 tests at t ≥ 2 should produce around 23 "discoveries" even with no edge anywhere.
• A good out-of-sample result is not proof of skill if the strategy was simply long a market that went up. Always compare with holding the market and with the same average exposure.
• Ask what the rule is really betting on — trend, time of day, mean reversion — and whether that would survive a different market regime.

Method, assumptions and limitations

• Data: Dukascopy XAU/USD 1-hour bid and ask bars, 2015-01-01 to 2026-10-16, 69,827 bars. In-sample: 2015–2021. Out-of-sample: 1 Jan 2022 to 2026-10-16.
• Rules: 1,000 distinct rules drawn at random (fixed seed) from seven families: moving-average crossovers (fast 2–60, slow up to 300 bars), RSI reversal and RSI trend (period 2–30, thresholds 10–40 / 60–90), Bollinger-band reversal (10–100 bars, 1–3 standard deviations), Donchian breakouts (10–300 bars), momentum (3–500 bars) and time-of-day holds (1–8 hours). Each rule was long/short, long-only or short-only at random. Nothing was optimised.
• Execution: signals on the bar close (mid price), position taken at the next bar's open — no look-ahead. One unit, no leverage, no compounding; results are the sum of hourly % returns, the same convention as our example checkup. Buy-and-hold is shown both the same way (sum of hourly %) and compounded.
• Costs: each change in position pays half the real bid/ask spread at that bar's open (median half-spread $0.17/oz) plus $0.10/oz slippage — the same as buying at the ask and selling at the bid. No commission. Overnight financing (swap) is not included; on most gold CFD accounts it would further reduce rules that hold positions for days or weeks.
• Significance: one-sample t-test on daily summed returns (t ≥ 2 ≈ one-sided p < 0.025). No multiple-testing correction is applied in the counts — which is the point of the exercise. Rules in the same family are correlated, so the "~23 by luck" figure is only a rough guide.
• Scope: one market, one timeframe, one split, during a decade in which gold rose strongly. Different markets, periods or rule families could give different results. This is research, not a recommendation of any rule.

The script, the full per-rule results and the summary output are kept on file and available on request.

Want this check run on your own strategy?

Send us your rules or your MT4/MT5 trade history. We reply with a plain-English report: sample size vs luck, out-of-sample, what is left after costs, and the worst case. Your strategy stays private unless you opt in.

• Get a free strategy checkup
• See a sample report

Research and education only — not investment advice and not a trading signal. Nothing here is a recommendation to use any strategy; any rules discussed were tested to show how backtests mislead, not to suggest trades. Past results do not guarantee future returns. Trading leveraged products such as gold CFDs can lose you more than you deposit. Full disclaimer · Privacy · Terms · X @ForwardProof_AI