Articles/Agents & Manager

◈ Agents & Manager/2026-09-01Intermediate

Write Hundreds of Files in One Burst, or Space Them Out?

When an agent generates hundreds of files, the slowness usually is not the writing. It is the rescans on the watcher side. I measured four write patterns so you can decide where to batch.

agent operations⁷ file watching performance¹³ Node.js⁶ workspace design

✦ Premium Article

The writing itself finished in 33 milliseconds.

That is the measured time for a routine that generates 400 files. The wait felt considerably longer than that, so the number made me suspect the delay was not happening on the side that writes the files. It was happening on the side that watches them.

Handing bulk generation to an agent is routine work now. Emitting article HTML. Exporting every icon size. Running a codemod across a few hundred files. All of them share the same experience: the write completes, and then things stay sluggish for a while.

I measured where that sluggishness comes from by varying how the files get written. Two of the results contradicted what I had expected going in.

The slow part is what happens behind the write

When you write a file, a notification goes out to every process watching that directory. Your editor, the dev server, the test runner, the agent's workspace index. Having several watchers on one directory at the same time is the normal state of affairs, not an unusual one.

The receiving side typically waits briefly before acting. The point of that pause is to collapse a run of consecutive notifications into a single rescan. That waiting period is the debounce window, and it is commonly set somewhere around 100 milliseconds.

Which means the amount of work you create for the other side is not decided by how many files you wrote. It is decided by whether your notifications landed inside the debounce window.

To measure that, write duration alone is not enough. You need to count how many rescans a watcher with a 100 millisecond window would end up starting.

The measurement script

This uses Node.js fs.watch to attach a watcher, then records event timestamps while varying the write pattern. Treating the debounce window as 100 milliseconds, every gap longer than that counts as one more rescan.

// bench.mjs — measure how the write pattern changes file-watch events
// usage: node bench.mjs <inplace|stage> <fileCount> <gapMs> <recursive:true|false>
//   e.g. node bench.mjs inplace 400 0 true
import fs from 'node:fs';
import path from 'node:path';
import os from 'node:os';
 
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const BODY = 'x'.repeat(2048); // 2KB per file; contents are not what we are measuring
const DEBOUNCE_NS = 100n * 1000000n; // 100ms in nanoseconds, to match hrtime.bigint
 
async function run(mode, count, gapMs, recursive) {
  // Rebuild the watched tree every run. Leftovers from a previous run skew the counts.
  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'watchbench-'));
  const target = path.join(root, 'out');
  fs.mkdirSync(target);
 
  let events = 0;
  const stamps = [];
  const watcher = fs.watch(target, { recursive }, () => {
    events++;
    stamps.push(process.hrtime.bigint());
  });
 
  const t0 = process.hrtime.bigint();
 
  if (mode === 'inplace') {
    // Write straight into the watched directory
    for (let i = 0; i < count; i++) {
      fs.writeFileSync(path.join(target, `f${i}.txt`), BODY);
      if (gapMs) await sleep(gapMs);
    }
  } else {
    // Build outside the watched tree, then move it in as one operation
    const stage = path.join(root, 'stage');
    fs.mkdirSync(stage);
    for (let i = 0; i < count; i++) {
      fs.writeFileSync(path.join(stage, `f${i}.txt`), BODY);
    }
    fs.renameSync(stage, path.join(target, 'batch'));
  }
 
  const t1 = process.hrtime.bigint();
 
  // Watch events arrive after the writes finish. Closing too early drops them.
  await sleep(1200);
  watcher.close();
 
  // Gaps wider than the debounce window = rescans the other side would start
  let rescans = 0;
  let last = null;
  for (const s of stamps) {
    if (last === null || s - last > DEBOUNCE_NS) rescans++;
    last = s;
  }
 
  console.log(
    JSON.stringify({
      mode,
      count,
      gapMs,
      recursive,
      events,
      rescans_100ms: rescans,
      write_ms: +(Number(t1 - t0) / 1e6).toFixed(1),
    })
  );
 
  fs.rmSync(root, { recursive: true, force: true });
}
 
const [mode, count, gapMs, recursive] = process.argv.slice(2);
await run(mode, Number(count), Number(gapMs), recursive === 'true');

One note on await sleep(1200). Watch events land after the write call returns. Calling watcher.close() immediately after writing drops the events still in flight and gives you a count lower than reality. My first numbers did not add up for exactly this reason, and it took me a while to see why.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦You will be able to tell whether a slow bulk-generation step is bottlenecked on writing or on the watchers observing your workspace

✦You will recognize, before you hit it, the condition under which slowing down your writes to be gentle actually makes things worse

✦You will be able to decide where generated output belongs in your workspace, taking the watcher's recursion mode into account

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Four write patterns, measured

Node.js v22 on Linux, 2KB per file.

Write pattern	Watcher	Files	Events	Rescans (100ms window)	Write time
All at once	recursive	400	400	1	33.7 ms
5ms apart	recursive	400	400	1	2,247.3 ms
120ms apart	recursive	40	40	40	4,837.1 ms
Stage elsewhere, then move	recursive	400	401	1	20.6 ms
Stage elsewhere, then move	non-recursive	400	1	1	14.7 ms
All at once	non-recursive	400	800	1	14.8 ms

The third row and the bottom two are where my expectations were wrong.

Spacing writes out gives the watcher more work, not less

Look at the third row. Forty files written 120 milliseconds apart produce 40 rescans. Four hundred files written in one burst produced one. A tenth of the files, forty times the rescans.

The reason is plain once you see it: 120 milliseconds is longer than the 100 millisecond debounce window. Every single write closes the window, and each time the watcher concludes that changes have settled and starts walking the tree. By the time the loop finishes, forty full traversals have piled up.

The 5 millisecond spacing in row two stays inside the window, so it still collapses to one rescan. The write takes 2.2 seconds, and the watcher's workload is identical to the one-burst case. Those 2.2 seconds do nothing for anybody.

This is the part that ran against my instincts. The considerate-looking move of writing slowly so as not to hammer the system becomes counterproductive the moment your interval crosses the window. If you cannot commit to keeping the gaps short, writing everything at once is the lighter option for the watcher.

Here is how I now think about the threshold.

Write interval	Effect on watchers	What to do
Comfortably below the debounce window	Collapses into a single rescan	Fine to write straight through
Around the debounce window	Collapsing becomes a coin flip	Either shorten deliberately or move output outside the watched tree
Above the debounce window	One rescan per write	The case to avoid. Batch the writes.

When an agent drives the generation, the interval between writes is dictated by model latency and tool-call round trips. An implementation that generates one file and immediately writes it lands squarely in that bottom row.

Staging outside the tree does not always help

Row four. I built 400 files in a temporary directory and moved them into the watched tree with a single rename. One operation, so I expected one event.

The measurement says 401 events — essentially one per file.

Recursive watching on Linux is implemented by attaching a separate watch to each directory. When a new directory appears inside the watched tree, the runtime walks its contents to add watches, and every file it finds along the way surfaces as a notification. Moving them as a unit does not change the fact that, from the recursive watcher's point of view, a large number of entries just appeared.

Row five is the same operation against a non-recursive watcher: one event. That watcher only observes the immediate children of the target, so all it sees is that a directory named batch now exists.

So the familiar trick of staging output and moving it in at the end cannot be judged effective without knowing how the other side attached its watch. For it to work, the destination has to sit outside any recursive watch.

The last row deserves a mention too. Writing 400 files directly into a non-recursive watcher produced 800 events, double the recursive case. File creation arrives as two separate notification types, rename and change. The same operation yields a different event count depending on how the watch was attached, which means event counts are only comparable when the recursion mode matches.

Measure the cost of one rescan against your own repository

Knowing the rescan count is only half the picture. You need the cost of a single rescan to make a decision. I measured against the repository behind a site I actually run.

// scan.mjs — time one recursive traversal
import fs from 'node:fs';
 
const dir = process.argv[2];
const t0 = process.hrtime.bigint();
const entries = fs.readdirSync(dir, { recursive: true, withFileTypes: true });
const t1 = process.hrtime.bigint();
 
console.log(
  JSON.stringify({
    dir,
    entries: entries.length,
    scan_ms: +(Number(t1 - t0) / 1e6).toFixed(1),
  })
);

Run against the directory holding this site's article sources:

Target	Entries	Time per traversal
content (article sources)	2,182	19.2 ms
public	25	0.6 ms

19.2 milliseconds for content. Negligible on its own, but multiply it by those 40 rescans and you get 768 milliseconds. That figure covers only the traversal itself, not the index updates or build triggers that follow it, and it multiplies again by the number of tools watching the same tree.

In an environment where dependency directories are also inside the watched tree, the entry count changes by an order of magnitude. I did not measure that case, but since traversal time scales with entry count, exclusion settings translate directly into wall-clock time. For verifying that your exclusions are actually taking effect, see checking why .antigravityignore patterns are not taking effect.

So how should the writes happen?

Based on the measurements, this is the order I now work through.

Ask first whether the output can live outside the watched tree. If it can, this is the most reliable answer. Generate everything, then move it in once, and you pay for a single rescan. Just remember that the effect disappears if the destination sits under a recursive watch. You can only claim it worked after checking the recursion mode.
If it cannot, write everything in one burst. Do not add pacing. Intervals that cross the debounce window are plainly harmful.
If generation is inherently one file at a time, batch only the writing. Let the agent generate at whatever pace it manages, accumulate results in memory or in an unwatched directory, and flush at the end. You may not control the generation rate, but you do control the write rate.
If it is still slow, count the watchers. In a typical indie developer setup, the editor, dev server, type checker, and test runner all share one machine and one tree, so the rescans happen once per watcher. Stopping the dev server for the duration of a generation run helps more than you would guess.

The lesson that stuck with me is that designing agent work means thinking not only about what the agent does, but about who is watching the result. Making the agent faster changes nothing if the shape of the work still produces 40 rescans downstream.

If your agent's search or file traversal is covering a different set of files than you expect, the conditions under which agent code search misses tracked files is worth reading alongside this, since watching and searching often disagree about what is excluded.

What to do next

Run the script once against the workspace you actually work in. Comparing node bench.mjs inplace 40 120 true with node bench.mjs inplace 400 0 true takes a few minutes and tells you which way the pacing question falls in your environment.

Deciding the write pattern from numbers settled the question far faster than tuning by feel ever did. I am still finding my way around a lot of this, but measuring what can be measured before deciding is a habit I intend to keep.

Thank You for Reading

Antigravity Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.