The bug that taught me to take this seriously was boring. A bot thought it was holding a position it had already closed, because the closing fill came back on a websocket message that got dropped during a reconnect. So it sat there, flat in reality, still hedging against a ghost. It kept placing orders to manage a position that did not exist, and every one of those was a small unnecessary fee and a bit of market impact. Nothing dramatic. Just a slow leak nobody noticed until I went looking for why my dashboard did not match the exchange.
Any bot that runs long enough will drift. Your internal ledger, the thing your code believes about what you hold, is a model of the exchange, and like every model it is wrong in small ways that compound. A reconciliation loop catches that drift on a schedule, decides what actually happened, and either fixes your state or wakes you up. It is unglamorous plumbing, and it is the difference between a bot you can leave running and one you have to babysit.
Where the drift comes from
It helps to name the sources, because each one wants a slightly different response. The common ones I have run into, roughly in order of how often they bite:
- Missed fills. A fill happens, the exchange knows, but your bot never got the message. Websocket dropped, your process restarted mid-fill, the notification arrived out of order. Your position is now wrong by a whole trade.
- Manual intervention. You, or someone with access, closed a position from the exchange app at three in the morning because it looked scary. The bot has no idea. This one is underrated as a cause of confusion, and it is almost always a human.
- Fee dust. Fees, funding payments on perps, and rounding at the exchange's quantity precision all shave tiny amounts off your balances. None of it is a bug. It just means expected and actual will never match to the last decimal, and if your loop treats that gap as an emergency you will get paged forever.
- Partial fills you rounded. Your order for 1.0 filled as 0.9997 plus a bit more later, and somewhere your code rounded. Now your notion of the position size and the exchange's notion diverge by a hair that grows every trade.
- Two writers. Another process, another bot, or a leftover cron job is touching the same account. This is rare but nasty, and reconciliation is often how you discover it exists.
The loop itself
Keep the mechanism simple. On a schedule, pull the truth from the exchange and compare it to what you think is true. The exchange is authoritative for what you hold. Your ledger is authoritative for why you hold it. Do not confuse those two, because it changes which side you trust when they disagree.
A run looks like this. Query current positions and balances over REST, not the websocket you have been trusting, because the whole point is to check that stream with an independent source. Snapshot your own expected state at the same moment. For each instrument, compute the difference between expected and actual, classify it into one of a few buckets, and only then decide what to do. The classify step is where the intelligence lives, and it is the step people skip.
How often to run it depends on how fast you trade. A market-making bot doing hundreds of orders a minute wants a light reconciliation every minute or two and a full one every few minutes. A slower strategy is fine reconciling every fifteen minutes. Either way, always reconcile on startup before you place a single order, since that is when your in-memory state is most likely to be stale. Startup reconciliation has caught more of my bugs than the scheduled kind.
Thresholds, or how to tell dust from a real problem
This is the part everyone gets wrong, usually by making it a single fixed number. A tolerance of 0.001 units is generous for Bitcoin and absurdly tight for a memecoin priced in fractions of a cent. The threshold has to be relative, and I usually think about it in two ways at once.
First, express the mismatch in the value of the position, not its raw quantity. A gap of a few dollars of notional on a five figure position is noise. The same few dollars on a fifty dollar position is a real fraction you should look at. Second, keep a separate absolute floor in account currency, because a tiny relative gap on a huge position can still be real money. A mismatch has to clear both the relative band and the absolute floor before it counts as a problem.
My rough rule of thumb, and you should tune it to your own fee schedule:
- If the gap is under roughly a tenth of a percent of position value and under a small fixed dollar floor, call it dust. Silently snap your ledger to the exchange and move on. Log it so you can watch the trend, but do not alert.
- If it clears both bands but you can explain it, a fill you can find in the trade history that your bot missed, treat it as a missed event. Replay that fill into your ledger, mark it reconciled, and alert at low severity so a human confirms it later.
- If it clears both bands and you cannot explain it, stop trading that instrument and alert loudly. An unexplained position gap means either someone else is touching the account or your accounting has a real bug, and you do not want to keep sizing new orders on top of state you no longer trust.
That third case is the one worth being paranoid about. When reality and your model disagree in a way you cannot account for, the safe default is to halt, not to auto-correct. Trusting the exchange feels right until the gap was caused by a bug that is still running, and now you have papered over the symptom while the cause keeps writing bad state.
Making the fix survive a restart
One detail that saves you real pain. When your loop decides to snap your ledger to the exchange, write that decision to durable storage as an explicit reconciliation event, not just an in-memory correction. If the process dies right after you correct in memory but before anything persists, you restart into the same wrong state and rediscover the same gap, except now your logs make it look like a recurring problem instead of a one-time fix. An append-only record of what you reconciled, when, and why also gives you the trail to reconstruct what a bot did during a bad hour.
The other thing worth building is a dust accumulator. Every time you snap away a sub-threshold gap, add it to a running total per instrument. Individually those corrections are noise. Summed over a week, a steadily growing total in one direction is a fingerprint of a rounding bug or a fee you are not accounting for, and it shows up long before any single reconciliation trips an alert.
None of this is exotic. It is a scheduled diff, a handful of thresholds, and a bias toward halting when you are confused. We lean on the same discipline inside Blockcircle for non-custodial execution across many venues, because the more exchanges you touch the more each one drifts in its own way. Build the loop early, before you have real size on, so that by the time it matters the boring plumbing is already there and tested.