What a backtest actually is
A backtest runs a trading strategy against historical price data to estimate how it would have performed. Define your entry and exit rules, feed in past prices, and the simulation tells you the hypothetical return, win rate, and drawdown. It's the standard way to evaluate a strategy before risking real money on it.
The appeal is obvious: it feels like evidence. A strategy that returned 340% over two years of backtested data looks a lot more credible than a strategy someone just believes in. That feeling is exactly the problem. Backtests are absurdly easy to make look good, often without anyone intending to cheat. The historical data is fixed and known, the strategy rules are flexible, and a motivated person testing enough variations will eventually find something that "worked" on that specific dataset — whether or not the underlying edge is real.
This is worth sitting with before looking at any backtest result, including ones this article will describe as reasonably rigorous. A backtest is a hypothesis test with a very high false-positive rate unless you go out of your way to control for it.
Survivorship bias: testing on the coins that lived
If you backtest a strategy using "the current top 200 coins by market cap" over the last five years, you've already broken the test. Most of the coins that existed five years ago aren't in the current top 200. Thousands of them are delisted, dead, or down 99.9% and forgotten. By selecting your universe from today's survivors, you've silently removed every coin that failed — and failure is exactly the outcome a risk strategy needs to be tested against.
Concretely: imagine a "buy low-cap alts with rising volume" strategy tested only against coins that are still trading today. It will look great, because by definition none of the coins in the test went to zero — the ones that did aren't in the dataset anymore. In reality, a meaningful fraction of low-cap alts bought on that signal historically went to zero or near it. The backtest never sees those trades because they're not in the survivor-only universe.
The fix is to build point-in-time universes: at each historical date, use the set of coins that actually existed and were tradable at that moment, including everything that later died. This is tedious and the data is harder to source than a simple current-day coin list, which is exactly why so many backtests skip it.
Look-ahead bias: using information from the future
Look-ahead bias means the backtest accidentally uses information that would not have been available at the moment the trade decision was made. It's subtler than survivorship bias and easier to introduce by accident.
A common version: a strategy rule says "buy if the daily close is above the 20-day moving average." If the backtest code decides this trade using the day's closing price, but simulates entering at some point earlier in that same day, it's using information (the close) that didn't exist yet when the trade supposedly happened. In live trading you don't know the close until the day is over.
Other common sources:
- Restated or corrected data. Some historical price feeds get revised after the fact (exchange outages get patched, erroneous prints get removed). A backtest using the cleaned-up data assumes a trader in the past had access to information that wasn't actually available then.
- Indicators computed with future bars. A moving average or volatility measure that's accidentally calculated using data points after the signal date.
- Delisting timing. If a coin gets removed from an exchange, a naive backtest might use its last available price as if that price could have been traded at, when in reality liquidity had already evaporated before delisting.
Look-ahead bias almost always makes backtests look better than reality, because the strategy is effectively cheating with a small peek into the future. It rarely makes results worse, which is part of why it's so easy to overlook — nothing about the output signals that something is wrong.
Overfitting: tuning until the past looks perfect
Overfitting, also called curve-fitting, happens when strategy parameters get adjusted repeatedly until the backtest result looks good on one specific historical dataset. Move the moving-average length from 20 to 23 days. Change the stop-loss from 8% to 6.5%. Add a filter that excludes trades on Tuesdays because those happened to lose money last year. Each adjustment individually seems reasonable — you're "optimizing." Collectively, you're sculpting a strategy that fits the noise in that specific historical sample rather than a real, repeatable pattern.
The tell is a strategy with many parameters, each tuned to a precise value, that produces an unusually smooth equity curve. Real market edges are noisy; a backtest with almost no losing months and a suspiciously high Sharpe ratio is more often a sign of overfitting than of skill.
A rough rule of thumb: the more parameters a strategy has, and the more of them were adjusted by looking at backtest output, the more skepticism the result deserves. A strategy with two or three parameters chosen from first principles (not tuned against the test data) is far more trustworthy than one with fifteen parameters grid-searched for the best historical outcome.
Ignoring the frictions that eat real returns
A backtest that assumes trades fill exactly at the signal price, with zero fees, is measuring a strategy that doesn't exist. Three costs get routinely underestimated or omitted:
- Trading fees. Even at 10 basis points per side, a strategy that trades frequently — say, 200 round trips a year — loses roughly 4% of capital annually to fees alone, before any consideration of whether the trades were profitable.
- Slippage. The price you'd actually get filled at, especially on lower-liquidity coins or during volatile moves, is worse than the quoted price at the moment of the signal. A strategy backtested at mid-price that would in practice need to cross a wide bid-ask spread on a thin order book can see its edge disappear entirely.
- Funding costs. For anything involving perpetual futures or leverage, funding rates accrue continuously and can be substantial during trending markets. A strategy that holds long positions through a period of consistently positive funding rates pays that cost every eight hours, and it rarely shows up unless it's explicitly modeled.
A strategy with a thin edge — say, a 2% average return per trade — can be fee-and-slippage-positive on paper and quietly break-even or negative once realistic frictions are included. This is the single most common reason a "profitable" backtest fails to translate into profitable live trading.
Small sample illusions: mistaking a regime for an edge
A strategy tested over six months of one specific market environment measures that environment, not the strategy. A momentum strategy backtested only during a strong uptrend will look excellent, because momentum strategies do well in strong uptrends — that's close to a tautology, not a discovery. The same strategy run through a choppy, range-bound six months would likely show a very different result.
Crypto markets cycle through distinct regimes: trending bull phases, grinding sideways ranges, high-volatility crashes, and low-volatility consolidations. A strategy needs to be tested across multiple regimes, ideally spanning at least one full cycle, before its backtested return means much of anything. A six-month test is a sample size of essentially one regime, and the statistics on a sample of one aren't statistics — they're an anecdote with a chart attached.
A related trap is treating a backtest's win rate as a precise number. A strategy that shows a 58% win rate over 40 trades has a wide enough confidence interval around that number that it could plausibly be a 45% or 68% win rate strategy that got unlucky or lucky in this particular sample. More trades, across more regimes, narrow that interval. Forty trades don't.
In-sample, out-of-sample, and walk-forward testing
The standard defense against overfitting is splitting the historical data into two parts. The in-sample period is the data used to design and tune the strategy — pick indicators, adjust parameters, decide on entry and exit logic. The out-of-sample period is data the strategy has never seen during development, used only once, at the end, to check whether the strategy still works.
If a strategy performs well in-sample but poorly out-of-sample, that's the signature of overfitting: the strategy learned the specific noise of the in-sample period rather than a general pattern. This single check catches a large fraction of curve-fit strategies before they go live.
Walk-forward testing extends this idea across time instead of doing it once. Instead of one in-sample/out-of-sample split, the strategy is repeatedly re-tuned on a rolling window of past data, then tested on the following unseen window, then the window rolls forward and the process repeats. A strategy tuned on 2021 data and tested on 2022, then re-tuned including 2022 and tested on 2023, and so on, gives a much better picture of whether the strategy's logic holds up as market conditions change — rather than whether it happened to fit one arbitrarily chosen historical stretch.
Walk-forward testing is more work to set up and it will almost always produce a less flattering result than a single well-tuned in-sample backtest. That's the point. A strategy's walk-forward performance is a far better estimate of what it will actually do with new money than its best-case in-sample number.
How much to trust any backtest, including this one
The realistic posture toward any backtest — yours, someone else's, or a track record published by a signal-scoring site — is to discount it substantially and ask specific questions before taking the number at face value: What universe of assets was tested, and does it include the ones that failed? Were fees, slippage, and funding modeled, and at what level? How many parameters were tuned, and against which data? Is the result out-of-sample or walk-forward, or is it the single best-looking configuration found after searching many? How many distinct market regimes does the test period actually span?
A composite score or signal framework that reports historical performance should be able to answer those questions plainly. If a track record doesn't specify its testing methodology, the honest assumption is that it hasn't controlled for these biases, not that it has. A backtest that survives all five checks and still shows a modest, unglamorous edge is far more trustworthy than one that shows a spectacular edge and doesn't mention how it was tested.