I pulled an exchange-reserve series a while back to test a simple idea, that outflows leading a rally are worth fading into and inflows are worth respecting as supply. The backtest looked great. Then I re-pulled the exact same series from the same provider a couple of months later, same date range, same asset, and the numbers were different. Not off by a rounding error. Different in the places where the trades actually happened. That is when it clicked that on-chain data is not a fixed historical record the way a candle chart feels like it is. It gets revised, quietly, all the time, and if you are not accounting for that your backtest is quietly reading tomorrow's newspaper.
Why on-chain numbers move after the fact
Raw chain data does not revise. A transaction that happened at a given block happened, and the hash is not going to change. What revises is everything built on top of the raw data, and almost every metric people actually trade off of is built on top of the raw data.
The key word is entity. Most useful metrics are entity-adjusted. Exchange reserves, netflows, miner balances, whale holdings, stablecoin supply on exchanges, all of these depend on a provider having correctly labeled which wallets belong to which entity. And entity labeling is a moving target. Exchanges spin up new deposit addresses constantly. They rotate cold storage. They rebrand, merge, or get acquired, and the clustering that connects a loose pile of addresses to a single named exchange is a heuristic that improves over time.
So when a provider discovers that a cluster of addresses it had been treating as unknown actually belongs to a large exchange, it does not just start counting them from that day forward. It typically backfills. It rewrites history so that those addresses were part of the exchange all along. Which is correct, in the sense that it makes the historical series more accurate. It is also exactly the thing that ruins your backtest, because the series you are now testing on includes knowledge that did not exist on the dates you are simulating.
Which metric families revise the most
Not everything revises equally. Roughly in order from most restated to least, here is how I think about it.
- Exchange reserves and exchange netflows. These are the worst offenders because they depend entirely on correct exchange wallet labeling, and exchange wallets are the most actively re-clustered category. A single newly discovered cold wallet can shift the historical reserve level by a meaningful chunk.
- Whale and cohort balances. Anything defined as balances held by wallets of a certain size, or by a labeled group, moves when the clustering underneath the cohort changes. Wallets get merged into or split out of an entity and the whole distribution shifts.
- Miner flows and labeled-entity flows. Same story. The flow is only as stable as the label on the sending and receiving side.
- Stablecoin metrics tied to exchanges. Supply on exchanges revises with exchange labeling. Total supply of a stablecoin, by contrast, is close to raw because it is just mint and burn events.
- Pure supply and activity counts. Active addresses, transaction counts, total supply, realized cap in its most basic form. These lean on raw data and revise the least. They still get methodology updates, but they do not get silently backfilled the way entity metrics do.
The rule of thumb I use is simple. The more a metric depends on someone deciding who owns a wallet, the more it will be restated, and the less you should trust a backtest built on it.
How this becomes lookahead bias
Lookahead bias is when your strategy makes a decision using information it could not have had at that moment. Revisions inject it in a sneaky way, because the timestamps all look correct. Every data point is stamped with an old date, so nothing screams that you are cheating. But the value at that old date was written recently, using clustering knowledge that only exists now.
Concretely, imagine a spike of outflows that, in the revised data, looks like a clean early signal before a big move up. In the version of the data that existed live, those addresses might not have been labeled as an exchange yet, so the outflow simply did not appear. Your live self would have seen nothing. Your backtest sees a beautiful signal. You fit a strategy to signals that were invisible in real time, and then you are surprised when the live version underperforms.
This is why on-chain strategies so often decay the instant they go live even when the backtest was long and the logic was sound. Part of the edge was never real. It was an artifact of testing on a cleaner, more complete, more retroactively-labeled dataset than any live trader ever had access to.
Building a point-in-time dataset, or faking one
The clean fix is a point-in-time dataset, meaning you store the value of each metric as it was reported on each date, not as it is reported now. If a provider gives you snapshots or an as-of query, use them. If you have to build it yourself, the discipline is to snapshot the series regularly and keep every vintage, so that months from now you can ask what the reserve number looked like on a given day rather than what it is now claimed to have been. It is tedious and it eats storage, but it is the only way to test honestly.
Most people cannot go back and reconstruct history they never saved, so here is the practical middle ground I actually use.
- Prefer raw-ish metrics. When two metrics express a similar idea, lean toward the one that depends less on entity labeling. A supply-based or activity-based proxy will be less flattering in backtest and far more honest going forward.
- Start saving vintages now. Even if you cannot rebuild the past, you can snapshot going forward. A few months of stored vintages lets you measure how much a given series actually revises, which is data you can put a number on.
- Measure the revision, then discount the edge. Take a metric, compare a vintage from a few weeks ago against the current version over the same dates, and look at how much the values moved right around your signal points. If they move a lot, haircut the backtested edge hard. If they barely move, trust it more.
- Add a lag buffer. Since revisions mostly add information you would not have had immediately, delaying your signal by a few days on the noisy metrics gets you closer to what was actually knowable live. It costs you some responsiveness and buys you a lot of realism.
- Stress-test on the ugly version. If you can find or reconstruct an earlier, less-labeled vintage, run the strategy on that. A strategy that only works on the pristine current data and falls apart on the older, messier version is telling you exactly what it is.
None of this makes on-chain data unusable. It just means treating the entity-adjusted metrics like economic data that gets revised rather than like price, which does not. Price at a timestamp is price. An exchange-reserve number at a timestamp is somebody's current best guess about who owned what, and that guess keeps getting better after the fact, which is wonderful for research and quietly dangerous for backtests. When a series looks too clean and the signals line up too well, my first move now is to pull an old vintage and see how much of that clean story survives.