← Research library
Research · 研究 · 24 · Methodology

Why most backtests lie. And how to read past it.

23 Jul 202610 min readMethodologyShishin Research

This article explains how backtests mislead and how to read past it. It is educational, about evaluating any track record, not personalised investment advice, and not a performance claim about any specific strategy.

A backtest is the easiest thing in finance to fake and the hardest thing to fake honestly. Almost any rule, tuned enough, can be made to produce a gorgeous historical equity curve, and almost none of those curves survive contact with live markets. The gap between the two is not bad luck. It is a small number of specific, mechanical errors, each of which inflates the past at the expense of the future. Knowing the errors is how you read a backtest for what it is rather than what it advertises.

Look-ahead bias: using information you wouldn’t have had

The most common and most lethal error is look-ahead: the backtest makes a decision using information that was not available at the moment of the decision. It is usually subtle. Using a day’s closing price to decide a trade you mark as entered at that day’s open. Using a company’s restated earnings rather than the numbers actually published at the time. Ranking a universe using a list of names that was only finalised later. Each one quietly hands the strategy a peek at the future, and even a tiny peek compounds into a fantasy curve.

Look-ahead is dangerous precisely because it doesn’t feel like cheating. The author isn’t trying to lie; the data pipeline leaks the future and the result looks brilliant. The only defense is a discipline of point-in-time data, every decision uses only what was knowable on the day, including which names even existed in the universe then. That last point is its own deep problem, the survivor leak, covered in survivorship bias.

Overfitting: fitting the noise instead of the signal

The second error is overfitting, tuning a strategy so precisely to the historical data that it captures the random noise of that specific period rather than any durable edge. Every added parameter, every threshold nudged to improve the curve, every “the strategy works except in these three months so let’s add a filter” is a step further into describing the past instead of predicting the future.

The tell is fragility. An overfit strategy’s performance collapses if you shift the test window by a month, change a threshold by a hair, or run it on data it wasn’t tuned on. A real edge is robust: it degrades gracefully when you perturb it, because it was capturing something structural, not a coincidence. A curve that is too perfect, no ugly stretches, every parameter suspiciously round, is more worrying than a good-but-scarred one.

The in-sample / out-of-sample split

The first defense against overfitting is to never judge a strategy on the data you built it with. Split history: develop and tune on one portion (in-sample), then test, once , on a portion the strategy has never seen (out-of-sample). If the edge survives on data it wasn’t fitted to, it has a chance of being real. If it only shines in-sample, it was memorisation.

The catch is that “once” is doing heavy lifting. The moment you peek at the out-of-sample result, dislike it, and go back to tweak, the out-of-sample data has become in-sample, you are now fitting to it too, just more slowly. Honest out-of-sample testing requires the discipline to accept a disappointing result rather than quietly tuning until it passes. That self-discipline failure is also why the bar for statistical significance has to be set far higher than the textbook one: when many variants are tried, some pass by luck alone.

Walk-forward: the closest a backtest gets to honest

Walk-forward validation is the more rigorous version of the split. Instead of one in-sample/out-of-sample division, you roll it: tune on a window, test on the next unseen window, then slide both forward and repeat across the whole history. The strategy is always being judged on data just after the data it was tuned on , which is exactly the situation it faces live, where you always trade the period just after the one you learned from.

Walk-forward doesn’t make a backtest truthful, nothing fully does, but it removes the easiest ways to fool yourself. A strategy that holds up walk-forward, across many rolling windows including ugly ones, is making a far stronger claim than one with a single beautiful in-sample curve. And the ultimate walk-forward is live: running the locked rules forward in public, where the next window is real and unseen by construction. That is why we paper-trade in public and why the day the live bot caught up to its backtest mattered: live is the test that can’t be overfit.

Reading a backtest you didn’t build

You usually can’t audit someone else’s pipeline, but a few questions surface most of the lies:

  • Is the curve suspiciously smooth? Real edges have ugly stretches. A flawless curve is a red flag, not a selling point.
  • How many parameters, and how round are they?A dozen finely-tuned thresholds is a description of the past. A few robust ones that work across a range is an edge.
  • Is there out-of-sample or walk-forward evidence?Or is the whole claim one optimised pass over all of history?
  • Is any of it verified forward? A backtest is a claim about the past. Live, public, unedited results are the claim being tested where overfitting can’t reach.

Most backtests don’t lie on purpose. They lie because the person who built them wanted the strategy to work and the data was happy to oblige. The defense is not cynicism, it is the small set of disciplines above, applied to your own work before you trust it, and demanded of anyone else’s before you do.

Sources & further reading

  • Bailey, D. H., Borwein, J. M., López de Prado, M. & Zhu, Q. J. (2014). “Pseudo-Mathematics and Financial Charlatanism.” Notices of the American Mathematical Society, 61(5), 458 to 471.
  • López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley.
Related reading
MethodologyThe setup state classifier: context beats raw signal8 min readMethodologyVolatility-aware stops: why fixed-percentage stops are the wrong tool8 min readMethodologyThe four guardians framework: why mythology, not factor labels7 min read
Frequently asked

Why do backtests overstate real performance?

Because it is easy to accidentally encode hindsight. Overfitting (tuning rules to past noise), look-ahead bias (using data that was not available at the time), and survivorship bias (testing only on names that survived) all inflate a backtest above what it would have earned live.

What is look-ahead bias?

Using information in the backtest that would not have been known at that moment, for example today's closing price to act at the open, or a company's later index membership. It makes a strategy look prescient and never reproduces live.

What is walk-forward validation?

Tuning on one window of history, then testing on the next unseen window, rolling forward, so every test is genuinely out-of-sample. That, plus live paper-trading, is the honest check a single in-sample backtest cannot provide.