When a Timed-Out Unattended Agent Leaves a Half-Written File Behind

Make the Write Atomic — Temp and Rename

The most reliable atomic write on a filesystem is the old discipline: write everything into a differently named temp file, then rename it onto the destination path. A rename (mv) on the same filesystem is atomic, so an observer of the destination path only ever sees either "fully written" or "not there yet." The in-between state is never visible from outside.

Wired into an unattended bash run, it looks like this.

#!/usr/bin/env bash
set -euo pipefail
 
# Destination, and a unique temp in the SAME directory
DEST="out/articles/result.json"
RUN_ID="$(date +%s)-$$"                 # seconds + PID = unique per run
TMP="$(dirname "$DEST")/.tmp.${RUN_ID}.json"
 
# Always clean up the temp, however we die
cleanup() { rm -f "$TMP"; }
trap cleanup EXIT INT TERM
 
# Have the agent write into the temp
agy run generate-article --out "$TMP"
 
# Validate what landed BEFORE publishing atomically
[ -s "$TMP" ] || { echo "empty output"; exit 1; }   # reject zero-byte
python3 -c "import json,sys; json.load(open(sys.argv[1]))" "$TMP"   # reject broken JSON
 
mv -f "$TMP" "$DEST"     # rename on same FS is atomic
trap - EXIT
echo "published: $DEST"

Three things carry the weight here. First, have the agent write into the temp — let it write the destination directly and that path is corrupted the instant it is killed. Second, trap ... EXIT INT TERM removes the temp no matter how the run ends; the timeout's SIGTERM is caught here too. Third, validate size and content right before mv, and only proceed to publish if validation passes.

In Node the idea is identical — slip an fsync in before the rename.

import { writeFile, rename, open } from "node:fs/promises";
import { dirname, join } from "node:path";
 
async function atomicWrite(dest, data) {
  const runId = `${Date.now()}-${process.pid}`;
  const tmp = join(dirname(dest), `.tmp.${runId}`);
  await writeFile(tmp, data, "utf8");
 
  // Flush the OS buffer to disk before renaming
  const fh = await open(tmp, "r");
  await fh.sync();
  await fh.close();
 
  await rename(tmp, dest);   // atomic on the same FS
}

The reason for fsync (fh.sync() in Node) is that even though rename itself is atomic, a power loss could in theory leave you with "the rename committed but the file's data has not reached the disk yet." For something running unattended around the clock, I prefer not to skip it.

Two Disciplines to Avoid Stepping on Last Run's Debris

Atomic writes eliminate the half-written file, but stale fixed-name temp contamination needs separate discipline. After getting burned by it, I now keep two rules without exception.

1. Never give a temp file a fixed name

A fixed name like /tmp/insert.txt is a breeding ground: the debris from a run that failed to write gets read verbatim next time. Give each run a unique name (seconds + PID, or a name that includes the target slug) as the code above does, and there is structurally no way for this run to pick up last run's temp. Also keep the temp in the same directory as the destination. Spanning two filesystems (say /tmp and your working directory) makes rename non-atomic — it degrades to copy-then-delete and the atomicity is gone.

2. Sweep old temps at startup

The trap handles the common case, but if the process is force-killed with SIGKILL (kill -9), even the trap does not run. So I add a safety net that sweeps my own temps older than a threshold at startup.

# At startup: remove abandoned temps older than 1 hour (only my naming scheme)
find out/articles -name '.tmp.*' -mmin +60 -delete 2>/dev/null || true

Restricting the sweep to "only files matching my own naming scheme" is essential. A broad pattern risks deleting an in-progress file from another agent working in the same directory.

Separate the Post-Write Content Assertion From the Exit Code

The final defense is content verification independent of the exit code. Before publishing with mv, confirm that what landed is what this run intended. Beyond rejecting zero-byte or broken syntax, I check run-specific markers: that this run's slug appears in the body, and that the previous run's slug did not sneak in.

# Just before publish: confirm this run's slug is present and no prior-run trace
grep -q "$SLUG" "$TMP"               || { echo "self-marker missing"; exit 1; }
grep -q "$PREV_SLUG" "$TMP" 2>/dev/null && { echo "previous-run contamination"; exit 1; }

There is one more operational call worth making. Do not fold verification and publishing into a single step. If you write "verify, and on pass mv in the same flow," a mid-step failure in the verification can be swallowed, and you proceed to publish unverified content. I keep verification as a gate I confirm has passed, then publish as a separate step. It is unglamorous, but across hundreds of unattended runs that separation earns its keep.

Where to Draw the Line

Almost none of this is needed for interactive runs. If you are watching the screen, you spot a half-written file immediately and delete it by hand. Atomic writes and content assertions truly pay off when the run is unattended and the output becomes the input of a later automated stage. Pipelines that chain artifacts through the Antigravity CLI or scheduled tasks are exactly that case.

Conversely, for a one-shot where a human inspects the result before moving on, all of this is overkill. My rule for whether to spend the cost is simple: does a later stage trust this output before anyone looks at it? Building pipelines as an indie developer, the temptation is to harden everything — but put the hardening in the wrong place and you have only added complexity.

Pick one artifact — the most downstream one nobody inspects — and replace just its write with temp-plus-rename. That alone should sharply cut the "wake up to a broken artifact" class of incident.