When a Scheduled Agent Runs Twice — Designing for Idempotency Against Overlap and Retry

A scheduled agent can do the same work twice when the next run triggers before the last one finishes. Here is a design with an overlap lock and an idempotency guard that survives mid-run failures, drawn from a double-publish incident I ran into in production.

agents⁸⁹ idempotency⁵ scheduled-execution³ locking reliability⁶

✦ Premium Article

I once found two nearly identical articles published within minutes of each other.

Because I run several sites on autopilot as an indie developer, I let scheduled agents handle content generation and report aggregation. One day generation ran heavier than usual and a single run overran its window. Before it finished, the next trigger fired, a second agent started, produced another article on the same topic, and pushed it.

We tend to assume a scheduled job "runs once at a fixed time," but in reality it can "run when the time comes even if the previous run hasn't finished." This article lays out an idempotency design for agents that survives both overlap and retry.

Why double execution happens

There are two main causes. If you do not separate them, your fix will miss.

The first is overlap. When the previous run drags on and the next trigger time arrives before it ends, two runs proceed in parallel. Generative agents vary widely in duration depending on input data and model response time — a job that usually finishes in five minutes occasionally takes twenty. A fixed-interval schedule does not absorb that jitter.

The second is retry. Most unattended execution environments automatically re-run failed tasks. The catch is "how far did it get before failing?" If the network drops right before the push, after the article file is already written, the run is judged a "failure" — yet the file remains, and re-running the same work duplicates the artifact.

These two differ in nature. Overlap is a "two run at once" problem, prevented with a lock. Retry is a "runs twice with a gap" problem, absorbed by making the artifact idempotent. You need both.

Prevent overlap with flock

The most reliable way to prevent overlap is to take an exclusive lock at the entry point of the run. Linux flock provides an advisory lock on a file descriptor and makes "give up immediately if the lock can't be taken" easy to express.

#!/usr/bin/env bash
# run_agent_guarded.sh
# If the same task is already running, skip this time
set -euo pipefail
 
LOCK="/tmp/agent-content-antigravity.lock"
 
exec 9>"$LOCK"
if ! flock -n 9; then
  echo "⏭ Previous run still in progress. Skipping this time ($(date '+%F %T'))"
  exit 0
fi
 
# From here, single execution is guaranteed
echo "▶ Started $(date '+%F %T')"
 
# --- actual agent work ---
generate_and_push_article
 
echo "■ Finished $(date '+%F %T')"
# Descriptor 9 closes automatically at process exit, releasing the lock

The -n (non-blocking) on flock -n is the crux. If the lock can't be taken it returns an exit code without waiting, so an overlapping run quietly yields to the next cycle. By not building a wait queue, you avoid runs piling up like a snowball. Put the lock file somewhere shared across processes such as /tmp; the descriptor releases on process exit, so no explicit release is needed.

That stops overlap. But a lock alone does not stop retry duplication, because two runs separated in time can each acquire the lock.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Separate and understand the two causes of double execution: overlap from a long-running previous run, and automatic retry after failure

✦Drop in an overlap guard with flock and an idempotency guard keyed on the work item, straight into your bash flow

✦Learn to design for at-least-once execution so that even a double run converges to a single artifact

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Absorb retry with an idempotency guard keyed on the work item

Idempotent means the result does not change no matter how many times you perform the same operation. Assume scheduled execution may run at-least-once but is not guaranteed exactly-once. On that basis, design so two runs converge to a single artifact.

In short, give the artifact you are about to create a unique "work key," and just before the push, check that "an artifact with that key does not already exist." For content generation, the slug derived from the topic is the work key.

# Idempotency guard: if this slug already exists in ja/en, stop the work
SLUG="antigravity-scheduled-agent-idempotency-overlap-guard-design"
JA="content/articles/ja"
EN="content/articles/en"
 
if find "$JA" "$EN" -name "${SLUG}.mdx" | grep -q .; then
  echo "⏭ slug=$SLUG already exists. Skipping generation."
  exit 0
fi
 
# Reaching here means the slug does not exist yet → safe to generate
write_article "$SLUG"

The key is to run the check not "at the start of the run" but "in the same place as the push decision." A check at the start can still be beaten by another run that creates the same slug while generation is slow. Placing the final check just before the push minimizes the window for a race.

To harden it further, check against the remote's latest state, not just local. If you run git pull --rebase before the slug check at the front of the scheduled run, you can also reject the case where another run on another machine already published the same slug.

Make the push itself idempotent

Even if you reject artifact duplication, push contention remains. When multiple runs push to the same origin/main, the latecomer is rejected with non-fast-forward. Treat that as a failure, rebase automatically, and retry exactly once.

push_with_rebase_once() {
  if git push origin main; then
    echo "✅ push succeeded"
    return 0
  fi
  echo "↺ Contention detected. pull --rebase and retry exactly once."
  git pull --rebase origin main || { echo "❌ rebase failed. Manual check needed."; return 1; }
  # After rebase, re-evaluate the idempotency guard (drop your copy if the same slug was published first)
  git push origin main
}

The trick is not to make it an infinite retry. If one rebase-and-retry still fails to land, that is likely not a simple race but another anomaly (permissions, a protected branch, a large divergence), so stop the machine's retries and log it.

The order of the design matters

The three mechanisms matter in the order you add them. First stop concurrent runs with flock, then reject artifact duplication with the work key, finally make the push idempotent with one rebase-and-retry. Layered in that order, even a double execution converges to a single artifact.

When designing a scheduled agent, my starting point is to discard the "runs exactly once" assumption and replace it with "does not break if it runs twice." The double-publish I opened with would have been prevented by a single flock. Start with the lock.

Thank You for Reading

Antigravity Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.