The better a backtest looks, the more suspicious I get. A beautiful equity curve with smooth, consistent profits should raise your guard rather than lower it. Most of the time those results are artifacts of how the test was built, not evidence of a real edge sitting underneath.
The four biases that inflate everything
Look-ahead bias creeps in when your backtest uses information that would not have existed at the moment of the trade. It is easier to introduce than people expect. Calculate an indicator across the full dataset instead of only the data available up to that bar, or lean on fundamentals that get revised weeks after the initial release, and you have quietly fed the future into the past. Both quietly inflate the results.
Survivorship bias shows up when your data only includes the assets that made it to the end. Backtest a stock momentum strategy on today's S&P 500 and you have already dropped every company that went bankrupt, got delisted, or was acquired somewhere along the way. Those are exactly the names a momentum strategy tends to buy high and ride to zero, so cutting them out is not a small rounding error, it is the difference between a real result and a fantasy.
Overfitting is tuning parameters to fit history at the expense of the future. Test 200 combinations, pick the best one, and you have not found a robust strategy. You have found the settings that happened to match the quirks of that one test window. The more knobs you turn, the more sure you can be the live results will look nothing like the chart on your screen.
Execution bias is the sneakiest of the four, because it is the hardest to see. A backtest assumes you fill at the printed price. In real life your orders move the price, and slippage, partial fills, and the bid-ask spread all drag actual returns below the backtested ones. For anything that trades often or touches illiquid markets, that gap can be brutal enough to swallow the whole edge.
Out-of-sample is the floor, not a bonus
The bare minimum for a backtest I will trust is out-of-sample validation. Split the data, build the strategy on the in-sample period, and test it on data it has never touched. The out-of-sample results are the only ones that count. If it shines in-sample and dies out-of-sample, it is overfit and you are done, no matter how good the first curve looked.
Walk-forward analysis pushes this further. Instead of one split, you roll the window forward. Build on years 1 through 3, test on year 4, then build on years 2 through 4, test on year 5, and keep going. You end up with a chain of out-of-sample results, which is a far more honest picture of how the thing holds up as conditions change. We lean on walk-forward in the Blockcircle backtester for exactly this reason.
Model the costs like you mean it
Your transaction cost assumptions make or break the entire exercise. At an absolute minimum, put in the bid-ask spread, exchange fees, and slippage. For crypto, add funding rates if you are carrying leveraged positions. For prediction markets, add the vig baked into YES plus NO pricing.
One quick stress test I like: double your cost estimates and see whether the strategy still makes money.
- If it survives doubled costs, the edge has some room to breathe once real execution shows up.
- If doubling costs kills it, the original edge was always too thin to make it through the real world anyway.
Read the degradation, not just the average
Watch how performance shifts as you move from older data toward recent data. Most edges fade as other people find the same pattern and trade it away. A strategy that did 30 percent a year from 2015 to 2018 and 10 percent a year from 2022 to 2025 might still be worth running, but you have to price in that slope instead of assuming the historical average will hold going forward.
Sudden drops worry me more than slow decay. If something worked for years and then stopped cold, a structural change probably killed the edge outright. Gradual decay means it is being arbitraged away slowly, which at least gives you time to adapt around it. An abrupt stop usually means the regime the edge depended on is simply gone.
The right amount of skepticism
Treat every backtest number as the ceiling, not the expectation. Live results come in worse, often 30 to 50 percent worse for moderate-turnover strategies once realistic costs are baked in. Apply that haircut first, before you get excited. If the strategy still looks good after the haircut, it is worth paper trading. If it only works at full backtest performance, leave it on the shelf and go find the next idea.