The first time cron cost me money it was a rebalance job that ran twice on a Sunday morning in the autumn. Nothing in my code changed. The clock did. My server was on a local timezone that fell back an hour, the 1:30am slot came around twice, and cron dutifully ran my job at both of them. The second run saw positions that were already balanced, decided they were slightly off because prices had moved, and traded again into a spread I did not need to pay. Small loss, but the kind that makes you sit up, because it means the thing you trusted to run once a day has a mode where it runs twice and you never told it to.
That is the whole problem with cron for trading jobs in one sentence. It is a beautifully simple tool that quietly assumes the world is simple too, and markets plus servers plus time are not. Here is how I think about the three ways it bites, and what I actually do about each.
Put everything on UTC and stop thinking about it
The DST double-run is not a cron bug. Cron did exactly what the crontab said, in the timezone the server was configured for. The fix is to remove the ambiguity entirely by running the machine, the cron daemon, and every timestamp in your job on UTC. UTC has no daylight saving. It never springs forward or falls back, so there is no repeated hour and no skipped hour, which also means there is no hour where a job silently does not fire because the wall clock jumped over it.
Set the server timezone to UTC. Log in UTC. Store times in UTC. If your strategy genuinely needs to act relative to a local market session, say the US equity open or a specific exchange's funding window, do that conversion inside the code at runtime using a real timezone library, not by tilting the whole box to New York time and hoping. The rule I follow is that the only place a local timezone is allowed to exist is the exact line of code that needs it, and everything upstream and downstream is UTC. It sounds rigid. It is rigid on purpose, because the alternative is debugging a phantom trade six months from now and slowly realizing the culprit was a clock.
One more thing that surprises people. Cron reads the schedule when it fires, so if you change the system timezone, restart the cron service. And test the fallback case deliberately by temporarily setting a machine to a DST-observing zone and watching what a 1:30am job does. You want to have seen the failure before it costs you.
Assume your job will overlap itself, because it will
The second failure is subtler and, for trading, more dangerous. You schedule a job every five minutes. It usually finishes in twenty seconds. Then one afternoon an exchange API gets slow, or a database query hangs, and your job takes eleven minutes. Cron does not know or care. At the next five-minute mark it starts a second copy while the first is still running. Now you have two processes reading the same signal, both deciding to open a position, and you have doubled your size by accident.
The defense is an overlap guard, and the simplest reliable one is a lockfile via flock. You wrap the job so it grabs an exclusive lock before doing anything, and if it cannot get the lock because a previous run still holds it, it exits immediately instead of piling on. On Linux this is one line in the crontab:
flock -n /tmp/rebalance.lock /path/to/rebalance.shruns the job only if no other instance holds the lock, and quietly does nothing if one does.- The
-nflag is the important part. It means fail fast rather than wait, because a trading job that queues up behind a stuck predecessor and then fires late is often worse than a job that skips a cycle.
A couple of things to get right. Put the lockfile somewhere that survives but is process-specific, and make sure a crashed job actually releases the lock. flock ties the lock to the file descriptor, so when the process dies the lock is released by the kernel, which is exactly what you want. Do not roll your own "check if a pidfile exists" version unless you enjoy the failure mode where the job crashes, leaves a stale pidfile, and then never runs again because it thinks a copy is always running. That one is a classic and it fails silent, which for a trading job means you find out when you check your positions and nothing has rebalanced in three days.
Decide what a missed run should do before it happens
Cron has no memory. If your VPS is down or rebooting at the moment a job was supposed to fire, that run is simply gone. It does not get queued, it does not run late, it evaporates. For a lot of jobs that is fine. For a daily rebalance or a funding-time hedge, a skipped run can leave you in a state you did not intend to hold.
The important decision is not technical, it is about intent. When a run is missed, do you want to catch up or skip? For an idempotent job that moves you toward a target state, like "bring the portfolio to these weights," catching up late is usually correct and safe, because running it at 9:05 instead of 9:00 just gets you to the same target a little later. For a job that acts on a specific moment, like "place this order at the exact funding timestamp," a late catch-up is often wrong and you should skip it and log loudly rather than fire a trade for a window that has already passed.
The way I build this is to stop trusting the schedule as the source of truth and instead have the job record its own last successful completion, then check on startup how long it has been. A rough version of the logic:
- On startup, read the timestamp of the last successful run.
- If the gap is within a normal cycle, run as usual.
- If the gap is large because the box was down, decide per job: for a state-convergence job, run once to catch up, not once per missed interval. For a moment-specific job, skip and alert.
- Write the new completion timestamp only after the work actually succeeds, so a crash mid-run does not falsely mark it done.
That last point matters more than it looks. If you stamp "done" at the start and then crash, the next run thinks everything is fine and you have silently skipped the actual work.
When cron has run out of road
Cron is genuinely good for a handful of independent jobs on one box. The moment you have jobs that depend on each other, need retries with backoff, need to backfill missed windows automatically, or need to run reliably across more than one machine, you are rebuilding a scheduler badly inside crontab and shell scripts. That is the signal to graduate to something with real state, like a job queue with a scheduler on top, or a workflow tool that tracks runs, retries, and dependencies as first-class things. Inside Blockcircle the scheduled work outgrew cron a while ago for exactly these reasons, and moving that logic into a proper task system removed a whole category of the failures above.
If you are still on a single VPS and cron is doing the job, that is completely reasonable. Just treat it as a tool that will run your thing twice, run two copies at once, or not run it at all, and put the three guards in before you find out which one it picks. Force UTC, wrap every job in flock -n, and make each job own its last-success timestamp so it can decide for itself what a gap means. The wiring is an afternoon. The trade it saves you is the one you never see.