The first time I tried to backtest a prediction market strategy, I did the lazy thing and treated a contract's price like a stock's price. Load the series, generate signals, mark to the close, plot the equity curve. The curve looked great. It was also completely fake, and it took me longer than I want to admit to see why. A prediction market contract is not a share of anything. It is a bet that pays one dollar if some event happens and zero if it does not, and almost every assumption baked into equity backtesting quietly breaks when you point it at a binary contract that resolves.
So this is the methodology I ended up with. It is less about clever signals and more about not lying to yourself with the mechanics. Get them right and a mediocre edge shows up honestly. Get them wrong and a nonexistent edge shows up looking like a fortune.
Odds data is not price data, and sourcing it is half the work
The cleanest way to think about a binary contract is that its price is a probability. A contract trading at 0.62 is the market saying roughly a sixty-two percent chance. That framing changes what you need out of your historical data. You do not just want the last trade. You want the full order book, or at least a decent bid and ask, at every timestamp you plan to act on, because the spread on these things is where most of your imagined profit lives or dies.
When you pull historical odds, get greedy about metadata. For every contract I want the resolution date, the actual resolution outcome, the market's open and close timestamps, and whatever unique identifier ties the contract back to the event so I do not accidentally stitch two different questions into one series. A lot of public datasets give you a midpoint tick history and nothing else, which is barely enough to fool yourself with. If you can get top of book with sizes, take it. The full book puts you in a much better position to model what your order would actually have done.
One quiet trap here is survivorship. If your dataset only contains markets that reached a clean resolution, you have silently dropped every market that got voided, relisted, or had its terms changed mid-flight. Those messy cases are exactly the ones a naive strategy loves to trade, so leaving them out flatters your results. Keep the ugly markets in and give the backtest an explicit way to handle a void.
Resolution and expiry are events, not just the end of a series
In an equity backtest, the end of the data is just where the chart stops. In a prediction market, the end is the whole point. The contract resolves to exactly zero or exactly one at a specific timestamp, and how you handle that timestamp determines whether your backtest is honest.
A few rules I now treat as non-negotiable. Any position still open at resolution settles at the true outcome, not at the last traded price, which is often a stale or thin print that has nothing to do with the payoff. Second, respect the resolution clock precisely. If news breaks and the market effectively knows the answer hours before official settlement, the price will already be sitting at 0.98, and a strategy that claims to buy at 0.60 in that window is buying a print no real order could have hit. Third, decide up front what a void does to a position. Usually it returns your stake, so your model should flatten the trade at cost rather than book a win or a loss.
The subtler version of this problem is lookahead through the resolution field itself. It is very easy to write a backtest where the resolution outcome leaks into a feature, because the outcome column is sitting right there in the same row. I keep the resolved outcome in a separate structure that the signal code physically cannot see, and only the settlement logic gets to touch it. If your signal function can read whether the event happened, your results are worthless and you will not notice.
Modeling slippage when the book is thin
This is the part that humbles people who came from liquid markets. Prediction market books are often thin, and the spread can be enormous relative to the edge you are chasing. A contract might show a midpoint of 0.55 with a bid at 0.51 and an ask at 0.59. If you backtest against the midpoint, you are assuming you can trade at 0.55. You cannot. To buy you pay 0.59, to sell you receive 0.51, and that eight cent round trip will eat a strategy that thought it had a three cent edge.
My rules of thumb for making slippage realistic:
- Fill against the far side of the book, not the midpoint. Buys hit the ask, sells hit the bid. This one change alone kills most strategies that looked profitable on midpoint data, and killing them on paper is the entire point.
- Cap your fill size at what was actually resting. If the best ask only had a small size, you do not get to fill a large order there at that price. Walk the book, or if you only have top of book, assume you move the price and add a penalty.
- Add a cost for being early or late. Near resolution the book thins out further and adverse selection gets worse, so I widen assumed slippage as expiry approaches instead of holding it constant.
- Treat maker and taker differently if the venue does. Sitting on the bid and getting filled is a different cost profile than crossing the spread, and pretending they are the same overstates a passive strategy.
If you take one thing from this section, let it be that backtesting on midpoint prices with a wide spread is the single most common way people convince themselves a losing prediction market strategy is a winner.
Scoring on calibration, not just return
Return matters, but for a probability-based strategy it is a noisy and misleading scoreboard. A strategy can get lucky on a handful of large, correlated events and post a great return while being badly miscalibrated underneath. What I actually want to know is whether my estimated probabilities line up with reality, and for that I lean on calibration scoring.
The workhorse is the Brier score, which is just the average squared difference between your predicted probability and the actual outcome, where the outcome is one or zero. Lower is better. A model that says 0.70 and is right most of the time scores well. A model that says 0.95 and gets surprised scores brutally, which is exactly the punishment you want for overconfidence. I also build a simple reliability check: bucket every prediction by its stated probability, then compare the average predicted probability in each bucket to the fraction that actually resolved yes. If your 0.80 bucket only resolves yes around sixty percent of the time, you are overconfident and no amount of return over a short sample changes that diagnosis.
One caution. Score calibration on the probability your model produced at decision time, not on some backfilled number. And be honest about correlation between events. Twenty contracts on the same election are not twenty independent bets, so treating them that way makes your calibration stats look far more solid than they are.
When I set up a new prediction market backtest now, I run through the same short list before I trust a single number it produces. Source odds with full resolution metadata and keep the voided and relisted markets in. Store the true outcome somewhere the signal code cannot reach. Fill against the far side of the book with size caps and widening slippage near expiry. Settle open positions at the real outcome and flatten voids at cost. Then score on Brier and a bucketed reliability curve before you even glance at the equity line.
Most of the strategies I have run through that pipeline died on contact, and that is the pipeline working. The ones that survive tend to be boring, edge that comes from being slightly better calibrated than a thin, distracted market rather than from any single dramatic call. When I want to sanity check the raw odds history and resolution timestamps before wiring up my own harness, pulling clean prediction market signal data through Blockcircle saves me the worst of the data janitorial work, but the discipline above is what actually keeps the backtest honest. Get the mechanics right first, then go looking for edge, not the other way around.