The One-Line Fix for Overlapping Cron Jobs

Cron has no memory. It fires every five minutes whether or not the previous run finished — and on the day your import takes seven minutes, you get two copies racing through the same rows.

Most teams reach for a Redis lock or a lock table in Postgres. On Linux, the fix already ships with your OS: flock, a mutex enforced by the kernel, wrapped around your command in one line.

Why this matters

Overlap is the default failure mode of scheduled jobs, and it only appears when you're least ready: a big payload Monday, a slow API, a 3 a.m. incident. The symptoms are ugly — duplicate emails, double-charged API calls, two backups writing to one file, exports interleaved mid-line.

It gets worse as you succeed. Busier systems mean longer job runtimes, which means higher odds of collision. And the hand-rolled fix — touch /var/lock/myscript.lock, delete on exit — leaks the moment a process is OOM-killed, and then your job never runs again. That's a worse bug than the one you fixed.

How it works

flock takes an exclusive lock on a lockfile, runs your command, then releases. The lock lives in the kernel, tied to an open file descriptor. When the process exits — cleanly, by crash, or by kill -9 — the kernel closes the fd and the lock evaporates. Stale locks are structurally impossible.

Three behaviors cover every case. Default: block until the lock frees. -n: if it's busy, exit immediately with status 1. -w 30: wait up to 30 seconds, then give up.

The cron line looks like this:

*/5 * * * * flock -n /run/lock/import.lock /usr/local/bin/import.sh

Every five minutes, cron tries to grab the lock. If the previous run still holds it, the new one exits silently. No queue, no state, no cleanup script.

Where this helps

• ETL and imports whose runtime spikes with data volume — the exact jobs most likely to overlap.
• Backup scripts: two pg_dump processes writing the same artifact will corrupt it.
• Anything that emails humans: duplicate reports and digests burn trust instantly.
• Docker entrypoints and CI retry steps — anywhere a script can be started twice concurrently.

Watch out

• It's Linux-only — flock comes from util-linux. macOS and stock BSD images don't have it.
• Locks are advisory. They only stop other flock callers on the same lockfile path. A rogue process ignoring the lock sails through.
• Don't put lockfiles in /tmp on shared machines — another local user can squat the name and DoS your job. Use /run/lock or an app-owned directory.
• flock coordinates one machine. Across hosts, you want a Postgres advisory lock or Redis — different tool, same idea.
• With -n, every skipped run exits 1, and cron will happily mail you about it. Append || true if skipping is normal behavior.

Try it yourself

Open two terminals and watch the second run get bounced:

# Terminal 1: hold the lock for 30 seconds
flock -n /tmp/demo.lock -c 'sleep 30'

# Terminal 2: exits instantly with code 1
flock -n /tmp/demo.lock -c 'echo ran' || echo "skipped: already running"

# The production pattern: skip instead of stack
*/5 * * * * flock -n /run/lock/import.lock /usr/local/bin/import.sh

TL;DR

• Cron starts a new run even if the old one is still going; flock wraps any command in a kernel-enforced mutex.
• The lock releases automatically on crash — no stale lockfiles, no Redis, no lock tables.
• Today: wrap your slowest cron job in flock -n and delete whatever lockfile hack it was using.