Articles/Agents & Manager

◈ Agents & Manager/2026-07-12Advanced

Clock Design for Agents: Separating Monotonic and Wall Time to Protect Long Runs

In scheduled and overnight agent runs, computing timeouts, retry backoff, and rate-limit windows from the wall clock breaks in quiet, hard-to-debug ways. This piece walks through a small Clock abstraction that splits monotonic from wall time, with a decision table and testable code.

agents¹⁴⁶long-runningmonotonic clocktimeout designscheduled executionreliability¹²

✦ Premium Article

One morning I opened the log of an agent I had left running overnight, and my hands stopped. A job that started around 2 a.m. had aborted itself with "timed out after 4 hours and 14 minutes." The actual work should have taken about ten minutes. After some digging, it finally clicked: the machine had slept partway through, and on wake the code treated the difference in wall-clock time as "elapsed time." The work was healthy. Only the way I read the clock was wrong.

As an indie developer building apps on my own, I hand a batch of maintenance chores to a scheduled overnight run, and this kind of clock-induced misbehavior is hard to avoid. The longer you entrust to an agent, the more that an unglamorous decision — how you measure time — quietly determines whether you can sleep while it works. This piece lays out a small design that deliberately separates a monotonic clock from a wall clock, with working code and a decision table.

Why elapsed time on the wall clock breaks

Let's start with the pattern most code adopts without thinking about it.

const start = Date.now();
await doWork();
const elapsedMs = Date.now() - start;
if (elapsedMs > 30_000) {
  throw new Error("work exceeded 30 seconds");
}

It looks perfectly reasonable, and for short work it rarely causes trouble. But Date.now() returns wall-clock time — a human-facing "what time is it right now." The wall clock moves for reasons that have nothing to do with your program.

Here are the situations where that difference drifts away from the real duration.

Event	Effect on the wall clock	What happens to elapsed-time math
NTP time correction	Jumps forward or back by hundreds of ms to seconds	Elapsed goes negative, or spikes
VM or laptop suspend/resume	Leaps forward by the time it was paused	Healthy work is misjudged as a timeout
Daylight saving transition	Shifts by one hour	Overnight jobs land off by an hour
Manual or sync time change	Jumps arbitrarily	Retry intervals and rate-limit windows break

The nastiest case is when time moves backward. If NTP pulls the wall clock into the past, Date.now() - start becomes negative, and elapsedMs > 30_000 never becomes true. The timeout never fires, and the agent waits forever. Conversely, right after resuming from suspend the difference balloons, and a job that hasn't finished anything gets cut off. Either way, the log only ever says "timeout" or "hung," and you burn hours before reaching the real cause.

The monotonic clock as an alternative

At the heart of the problem is reusing a clock meant for "displaying the current time" to "measure how long something took." For durations, the right tool is a monotonic clock.

A monotonic clock is guaranteed never to run backward. It advances only forward, whether NTP corrects the system time or DST flips. In Node.js that is process.hrtime.bigint(); in browsers and Deno it is performance.now(). What it returns is not a "when" but a relative "how far from some origin." You can't read it as a human timestamp, but take the difference of two readings and you get an accurate duration.

const start = process.hrtime.bigint();
await doWork();
const elapsedMs = Number(process.hrtime.bigint() - start) / 1_000_000;
if (elapsedMs > 30_000) {
  throw new Error("work exceeded 30 seconds");
}

The visible change is tiny, but the meaning shifts a great deal. This elapsedMs reflects the real time that actually passed, no matter how the machine slept or how NTP thrashed. The catch is that a monotonic value is relative to an origin, so it can't be used across processes or written to a log as a human-readable time. That is exactly where a division of responsibility is needed.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Why timeouts, backoff, and rate-limit windows silently break when measured on the wall clock during VM suspend or NTP correction

✦A small Clock abstraction that routes durations to a monotonic clock and timestamps to the wall clock

✦A decision table for which clock to use per purpose, plus how to test timing logic deterministically

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

A small Clock abstraction that splits roles

The shape I settled into across several scheduled tasks was to split the clock by purpose and gather it into an explicit abstraction. Durations use the monotonic clock; records and next-run schedules use the wall clock. I draw the boundary there.

export interface Clock {
  // monotonic: only for durations, timeouts, backoff
  monotonicMs(): number;
  // wall: only for logs, storage, human times, next-run schedules
  wallNow(): Date;
}
 
export const systemClock: Clock = {
  monotonicMs: () => Number(process.hrtime.bigint() / 1_000_000n),
  wallNow: () => new Date(),
};
 
// hold a deadline as a point on the monotonic clock
export function createDeadline(clock: Clock, budgetMs: number) {
  const deadline = clock.monotonicMs() + budgetMs;
  return {
    remainingMs: () => deadline - clock.monotonicMs(),
    expired: () => clock.monotonicMs() >= deadline,
  };
}

The key is holding the deadline as "a point reached on the monotonic clock," not "some hour and minute on the wall clock." With that, remainingMs() and expired() are immune to suspend and time correction. In an agent's step loop, you thread the deadline through like this.

async function runSteps(clock: Clock, steps: Step[], budgetMs: number) {
  const dl = createDeadline(clock, budgetMs);
  for (const step of steps) {
    if (dl.expired()) {
      // use the wall clock for records (a human reads it later)
      log.warn(`stopped over budget: ${clock.wallNow().toISOString()}`);
      break;
    }
    await step.run({ remainingMs: dl.remainingMs() });
  }
}

Only the timestamp in the log uses wallNow(), the wall clock. That line exists so a human can read "when it happened," and a relative monotonic value would be useless there. The two clocks don't compete — they divide the work by role.

Retry backoff belongs on the monotonic clock too

Alongside timeouts, the other place the wall clock sneaks in is retry backoff. Remembering "the time of the last failure" on the wall clock and deciding the wait from "the difference to now" collapses the moment time moves backward: every retry stampedes into the same second, or the next retry never arrives.

async function withRetry<T>(
  clock: Clock,
  fn: () => Promise<T>,
  opts: { maxAttempts: number; baseMs: number },
): Promise<T> {
  let attempt = 0;
  while (true) {
    const attemptStart = clock.monotonicMs();
    try {
      return await fn();
    } catch (err) {
      attempt++;
      if (attempt >= opts.maxAttempts) throw err;
      // exponential backoff + jitter, measured on the monotonic clock
      const spent = clock.monotonicMs() - attemptStart;
      const backoff = opts.baseMs * 2 ** (attempt - 1);
      const jitter = Math.random() * opts.baseMs;
      const waitMs = Math.max(0, backoff + jitter - spent);
      await sleep(waitMs);
    }
  }
}

Here too the basis for every decision is monotonicMs(). Subtracting the time already spent on the attempt (spent) from the backoff keeps the real interval to the next attempt stable, whether a failure came fast or slow. The same holds when you manage a rate-limit window yourself (say, "10 calls per 60 seconds"): mark the window's start on the monotonic clock and, even if time jumps, you avoid double-counting and dropped counts.

A decision table by purpose

Which clock to use for which purpose is worth tabling once. Whether you're writing new code or reviewing it, this at-a-glance view speeds up the judgment.

Purpose	Clock	Reason
Timeout check	Monotonic	Only real elapsed time matters; must not follow time corrections
Retry backoff	Monotonic	Wait intervals are relative; backward time collapses them
Rate-limit window	Monotonic	Window length is relative; prevents double counts and drops
Step budget / deadline	Monotonic	Keeps the budget across a suspend
Log / audit timestamp	Wall	Information for a human reading "when"
Next scheduled run	Wall	Tied to the calendar ("daily at 7"); be explicit about timezone and DST
Deadline across processes	Wall (recomputed to monotonic)	Monotonic values can't be shared; pass a wall time, let the receiver convert

That last row is where practice trips people up. A monotonic value only has meaning within the same process. When you hand a job queue "please finish by this deadline," pass the absolute wall time (ideally UTC), and have the receiving worker compute "how many ms remain" and convert it to a deadline on its own monotonic clock. Switching clocks between the moment you pass it and the moment you measure it is the one bit of care that keeps a budget intact across distributed steps.

Testing it deterministically

The biggest practical payoff of making Clock an abstraction is testability. In place of systemClock, you can inject a fake clock that advances time on your terms.

function fakeClock(startMono = 0, startWall = "2026-07-12T10:00:00Z"): Clock & {
  advance(ms: number): void;
} {
  let mono = startMono;
  let wall = new Date(startWall).getTime();
  return {
    monotonicMs: () => mono,
    wallNow: () => new Date(wall),
    advance(ms) { mono += ms; wall += ms; },
  };
}
 
// test: advancing only the monotonic clock (simulating suspend) still honors the deadline
const clock = fakeClock();
const dl = createDeadline(clock, 30_000);
clock.advance(29_000);
console.assert(!dl.expired(), "29s should not be expired");
clock.advance(2_000);
console.assert(dl.expired(), "31s should be expired");

Push a little further and let advance move the monotonic and wall clocks separately, and you can reproduce the mean cases too: "only the wall clock moves back" (NTP correction), "only the wall clock leaps" (time sync after resume). The "4-hour false timeout" that vexed me at the start reproduces in a few lines with tests shaped like this, and can be pinned so it never breaks again. Tests that rely on real waiting tend to be slow and flaky; a design where the clock is injectable lets you verify time-related logic instantly and deterministically.

A small habit that pays off in long runs

A recent Antigravity update stores OAuth tokens in the OS keyring automatically, cutting how often you hit an auth prompt. That is a step toward running agents longer without interruption. Which is exactly why, as run times stretch, the way you read the clock decides more outcomes. It would be self-defeating for a job that no longer stalls on auth to abort itself over a misread clock instead.

Adopting this doesn't have to be a large effort. Start by inventorying the Date.now() calls in existing code by purpose. Ask of each one, "is this measuring a duration, or is it a timestamp for the record?" and move the former to a monotonic clock. That alone erases most of the suspend- and NTP-induced misbehavior. For me, a single pass of that inventory brought my overnight tasks' mysterious aborts to a clean stop.

A clock is the kind of foundation that usually stays below conscious notice. But once you hand long stretches of time to a machine, the soundness of that foundation ties directly to whether you can rest easy. Slip in one small abstraction, and you can tuck the uncertainty around time into a quiet corner of the code. I hope this makes your overnight runs a little quieter. Thank you for reading.

Thank You for Reading

Antigravity Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

✦Copy-paste ready implementation code
✦New advanced guides published daily
✦$5/mo or $10 for lifetime access

View Membership →