Capping Parallel Agents With a Token Budget — Designing a Guard That Stops Runaway Cost

Allocating a budget per task

Allocation first. I put "run" at the top level, with multiple "tasks" hanging beneath it. The key is to hold a ceiling for the whole run and a separate ceiling for each task. A run-only ceiling cannot stop a single runaway; a task-only ceiling lets the total go unbounded as parallelism grows.

// budget.ts — budget allocation definitions
export type Money = number; // USD. Everything internal is unified in USD
 
export interface BudgetPolicy {
  runCeiling: Money;        // upper limit for this whole run
  taskCeiling: Money;       // upper limit a single task may use
  softRatio: number;        // start degrading past this fraction (e.g. 0.8)
}
 
// Per-model unit price (per 1M tokens, USD). Keep in sync with real billing
export const PRICE_TABLE: Record<string, { input: Money; output: Money }> = {
  "fast":     { input: 0.30, output: 1.50 },
  "balanced": { input: 3.00, output: 15.0 },
  "deep":     { input: 8.00, output: 40.0 },
};
 
export function costOf(model: string, inTok: number, outTok: number): Money {
  const p = PRICE_TABLE[model] ?? PRICE_TABLE["balanced"];
  return (inTok / 1_000_000) * p.input + (outTok / 1_000_000) * p.output;
}

Always externalize the price table. Model prices get revised, and the model you use differs by site and by purpose. In my case article generation uses balanced, simple work like link audits uses fast, and only design review uses deep — I vary the allocation by the nature of the task.

There is also a reason everything is unified in USD internally. I tend to think in yen by site, but model prices are usually quoted in dollars, and if conversion formulas scatter across the codebase, you will get a digit wrong somewhere. In my first implementation I held some values in yen and some in dollars, and the ceiling comparisons became meaningless. The unit of the values you compare should be pinned in one place.

Accumulating real spend in real time

Next is metering, and this was the crux. The essence of the accident was "I could only learn consumption after the fact," so I keep a ledger that holds the running total while the job is live.

// ledger.ts — a ledger that accumulates token spend in real time during a run
import { costOf, Money } from "./budget";
 
interface Entry {
  taskId: string;
  model: string;
  inTok: number;
  outTok: number;
  cost: Money;
  at: number;
}
 
export class TokenLedger {
  private entries: Entry[] = [];
  private byTask = new Map<string, Money>();
  private runTotal: Money = 0;
 
  record(taskId: string, model: string, inTok: number, outTok: number): Money {
    const cost = costOf(model, inTok, outTok);
    this.entries.push({ taskId, model, inTok, outTok, cost, at: Date.now() });
    this.byTask.set(taskId, (this.byTask.get(taskId) ?? 0) + cost);
    this.runTotal += cost;
    return cost;
  }
 
  taskSpend(taskId: string): Money {
    return this.byTask.get(taskId) ?? 0;
  }
 
  total(): Money {
    return this.runTotal;
  }
 
  // emit line items for later reconciliation
  dump(): Entry[] {
    return [...this.entries];
  }
}

The important part is to run record immediately after every LLM call. Even running in parallel, as long as the ledger instance is shared, runTotal sums consumption across all agents. At first I split the ledger per agent and could not see the whole-run total — a silly implementation. "Collect values you want to sum in one place" sounds obvious, but it surprisingly breaks once you go parallel.

The need for real time may seem excessive at first. But with ten running in parallel, no one can predict when the sum will hit the ceiling. If one task suddenly gets heavy, the whole total jumps even if the others are light. After-the-fact aggregation only gives you the number after the jump. Because you accumulate while running, the judgment "we are now at 80% of the ceiling overall" holds, and you can act while there is still room to stop. I feel this single point — seeing the running total while in flight — is the greatest value of the governance layer.

Always save the dump() line items after a run. The hardest part of the opening accident was not knowing which task was the culprit. With line items kept, I can pinpoint the cause in five minutes the next morning. It pays off as insurance.

A hard cap with staged degradation

The third layer is cutoff. With allocation (budget) and metering (ledger) in place, I implement a BudgetGuard that reconciles the two and makes a call. The call has three stages.

Stage one is normal: do nothing until the soft threshold (e.g. 80% of the ceiling). Stage two is downgrade: once past the soft threshold, automatically switch subsequent calls to a cheaper model. Finish quality drops a little, but in many situations that beats stopping. Stage three is halt: at the hard cap, allow no further calls for that task (or the whole run).

// guard.ts — reconcile budget and ledger, decide on degradation
import { BudgetPolicy } from "./budget";
import { TokenLedger } from "./ledger";
 
export type Decision =
  | { action: "proceed"; model: string }
  | { action: "downgrade"; model: string }
  | { action: "halt"; reason: string };
 
export class BudgetGuard {
  constructor(
    private policy: BudgetPolicy,
    private ledger: TokenLedger,
  ) {}
 
  // run before every call; follow the returned action
  check(taskId: string, requestedModel: string): Decision {
    const run = this.ledger.total();
    const task = this.ledger.taskSpend(taskId);
 
    // hard cap: halt if either the whole run or the task exceeds
    if (run >= this.policy.runCeiling) {
      return { action: "halt", reason: `run ceiling $${this.policy.runCeiling} reached` };
    }
    if (task >= this.policy.taskCeiling) {
      return { action: "halt", reason: `task ${taskId} ceiling reached` };
    }
 
    // soft threshold: downgrade once run OR task spend passes softRatio
    if (
      run >= this.policy.runCeiling * this.policy.softRatio ||
      task >= this.policy.taskCeiling * this.policy.softRatio
    ) {
      return { action: "downgrade", model: "fast" };
    }
 
    return { action: "proceed", model: requestedModel };
  }
}

Applying the soft threshold per task, and not only to the run as a whole, is something I fixed later. The first implementation downgraded on run-level spend only. That meant in the most common situation — the run still has headroom, but one task is heavy — that task stayed on the expensive model right up until it slammed into taskCeiling and halted outright. The downgrade stage I had built precisely to avoid abrupt stops never fired for a single runaway. I had written a whole section about getting burned by a halt-only design, and the code was effectively halt-only for individual tasks.

To see what adding task >= taskCeiling * softRatio actually changes, I lifted the check above verbatim and ran it locally. Settings are runCeiling: 4.0, taskCeiling: 0.8, softRatio: 0.8, with a single heavy task consuming 40,000 input and 4,000 output tokens per call — roughly $0.18 a call on the expensive model.

Nearly the same money, twice the distance. Before the fix, the task ran four calls on the expensive model and hit taskCeiling just before the fifth, never passing through a downgrade at all. After, it reaches 80% of its own ceiling after the fourth call, drops to the cheaper model, and buys the rest of its runway there. The halt still happens — what changes is how far the task gets on the same budget. Downgrade is not a savings mechanism so much as a way to bank some output before the cut-off arrives.

Run check before a call. record is after. Keeping this before/after symmetry is the trick — forget either and budget leaks. I folded "guard, then call, then record" into a single function so agents cannot touch the pieces individually.

// guarded-call.ts — the sole entry point that always pairs check with record
import { BudgetGuard } from "./guard";
import { TokenLedger } from "./ledger";
 
export async function guardedCall(
  guard: BudgetGuard,
  ledger: TokenLedger,
  taskId: string,
  model: string,
  invoke: (model: string) => Promise<{ inTok: number; outTok: number; text: string }>,
): Promise<{ text: string; halted: boolean }> {
  const decision = guard.check(taskId, model);
  if (decision.action === "halt") {
    console.warn(`[budget] halt ${taskId}: ${decision.reason}`);
    return { text: "", halted: true };
  }
  const useModel = decision.action === "downgrade" ? decision.model : model;
  const res = await invoke(useModel);
  ledger.record(taskId, useModel, res.inTok, res.outTok);
  return { text: res.text, halted: false };
}

Now, however the agent is implemented, as long as the path to the LLM goes through guardedCall, the budget always applies.

Conversely, leave even one bypass and budget leaks through it. When I review agent code, the one thing I always check is that no raw call equivalent to invoke is written outside guardedCall. Narrowing the entry point to one is unglamorous, but it is the single most effective rule for turning governance from "operating carefully" into "operating with structural protection."

This staging may look excessive. But I once tried a halt-only design and got burned. A task like article generation does most of its lookup up front and finishes at the end, so 80% of spend lands in the first half. If it touches the threshold and halts immediately in the first half, it gets cut off in a half-finished state, wasting the night's tokens entirely. Inserting downgrade as a middle stage lets you choose "finish with a cheaper model, but finish." The output is not perfect, but it beats zero. Protecting cost and not zeroing out the result are two goals worth thinking about separately.

Wiring it into Antigravity's parallel agents

Now to fold the three layers into Antigravity's parallel agents. Antigravity lets you run multiple sub-agents at once, but each calls the LLM independently, so left alone the combined consumption is invisible. So I create the ledger and guard exactly once at run scope and hand the same instance to every sub-agent.

// orchestrate.ts — distribute a shared guard to parallel agents
import { TokenLedger } from "./ledger";
import { BudgetGuard } from "./guard";
import { guardedCall } from "./guarded-call";
 
async function runDailyJobs(tasks: { id: string; model: string; run: Function }[]) {
  const ledger = new TokenLedger();
  const guard = new BudgetGuard(
    { runCeiling: 4.0, taskCeiling: 0.8, softRatio: 0.8 },
    ledger,
  );
 
  // the point: every task shares the same ledger / guard
  const results = await Promise.allSettled(
    tasks.map((t) =>
      t.run((model: string, invoke: any) =>
        guardedCall(guard, ledger, t.id, model, invoke),
      ),
    ),
  );
 
  // persist line items after the run (cause analysis takes 5 minutes)
  await persistLedger(ledger.dump(), new Date());
  console.log(`[budget] run total: $${ledger.total().toFixed(3)}`);
  return results;
}

Antigravity's parallel agents leave a work log and artifacts per sub-agent. I write the ledger's run total, and whether a downgrade or halt fired, into those. With that in place, opening the artifact the next morning shows the night's cost and how many times the guard kicked in at a glance. The agent's output and the cost record sit in the same place, so I can review quality and cost together rather than separately — which I like.

Using Promise.allSettled here is deliberate, though the reason deserves to be stated precisely. guardedCall does not throw on halt — it returns halted: true — so a budget stop is not itself a rejection. What allSettled protects against is a task that receives the stop, decides it cannot continue from a partial result, and throws on its own, plus the ordinary exceptions from network and parse failures. Under Promise.all, any one of those would drag down and discard every healthy task's result alongside it.

If you would rather have stops surface as failures, throw a dedicated error type from guardedCall instead. I chose to receive it as a return value, because the partial output of a halted task is still something I want to look at the next morning.

I put the run-scope runCeiling at $4.0 and the per-task ceiling at $0.8 to match one night's job size for my six-site operation. I derived them from the billing delta of the first accident. Adjust them to your own scale.

runCeiling gets overshot by however many agents are in flight

After a few nights of running orchestrate.ts, I noticed that the ledger total had clearly passed the $4.0 ceiling on some of them. The logs showed the guard returning halt exactly as designed. And the total still went over.

The gap sits between check and record. What check reads is ledger.total(), and that number does not include the calls that have not come back yet. If ten parallel tasks hit check in the same instant, all ten see the same comfortable total and all ten receive proceed. The ceiling gets crossed afterwards, by however many calls had already left the gate.

This is a different problem from the concurrent ledger writes I describe further down. What races here is not the Map update but the decision taken across an await, so it happens on single-threaded Node too.

I pulled the check and record ordering into a small script on my machine and varied the weight of a single call. runCeiling: 4.0 and softRatio: 0.8 are the same values used above.

With light calls this reads as a rounding error. The heavier each call gets, the more the overshoot is simply parallelism multiplied by unit price. In the bottom row, six calls go out together without a single halt between them and burn 2.4x the ceiling. The shape of my original accident was still there, inside the guard that was supposed to prevent it.

A ceiling has to be drawn against calls that have left, not against spend that has settled. The fix is plain: when check clears a call, put its estimate on the ledger as a reservation, then drop the reservation and swap in the real figure at record time.

// appended to ledger.ts — hold in-flight calls as reservations
export class TokenLedger {
  // ...existing fields unchanged
  private reserved: Money = 0;
  private holds = new Map<string, Money>();
 
  // called once check clears; reserves the estimate and returns a receipt
  reserve(model: string, estIn: number, estOut: number): string {
    const est = costOf(model, estIn, estOut);
    const holdId = `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
    this.holds.set(holdId, est);
    this.reserved += est;
    return holdId;
  }
 
  // paired with record; drops the reservation, leaving only the real figure
  release(holdId: string): void {
    const est = this.holds.get(holdId);
    if (est === undefined) return;
    this.holds.delete(holdId);
    this.reserved -= est;
  }
 
  // this is what the guard should read: settled spend + estimates in flight
  committed(): Money {
    return this.runTotal + this.reserved;
  }
}

Swap the two this.ledger.total() reads inside BudgetGuard.check for this.ledger.committed(), and rework guardedCall to reserve first.

// guarded-call.ts — reserve, call, then release and record the real figure
const decision = guard.check(taskId, model);
if (decision.action === "halt") {
  console.warn(`[budget] halt ${taskId}: ${decision.reason}`);
  return { text: "", halted: true };
}
const useModel = decision.action === "downgrade" ? decision.model : model;
const holdId = ledger.reserve(useModel, ESTIMATED_IN, ESTIMATED_OUT);
try {
  const res = await invoke(useModel);
  ledger.record(taskId, useModel, res.inTok, res.outTok);
  return { text: res.text, halted: false };
} finally {
  ledger.release(holdId);   // never leave a reservation behind on a throw
}

The release belongs in finally for a specific reason. If a call dies on a network exception and its reservation stays on the ledger, committed() never comes back down, and every task after it receives a halt for no reason at all. Meaning to stop spend and instead stopping the service through a missed cleanup path is not a trade I want to make.

Rerunning the same script with reservations in place brought all three setups in at $4.01–$4.04, within one percent of the ceiling. The overshoot does not reach zero, because the one call already past check when the reservation lands still goes out. The bound moves from "parallelism times one call" to "one call." That is the whole change.

For ESTIMATED_IN and ESTIMATED_OUT I use the maximum of the last twenty calls, not the median. A median makes the reservation thinner than reality, and the overshoot comes back in miniature. Anything I over-reserve is replaced by the real figure at record time, so a generous estimate costs nothing — it only downgrades a little sooner. Estimating high and reconciling afterwards has been the calmer order of operations for me.

A naive parallel run vs. one routed through the guard

Let me put the before and after side by side in code. Back when I was having accidents, the naive implementation looked roughly like this.

// Before — no guard. Consumption is only knowable from the post-run bill
async function runDailyJobs(tasks: { id: string; run: Function }[]) {
  // each task hits the LLM however it likes; no sum, no ceiling
  return Promise.all(tasks.map((t) => t.run()));
}

At a glance there is nothing wrong with it, and most days it runs quietly. But with no ceiling anywhere, a single runaway is unstoppable, and you only notice the anomaly once the bill settles.

Routing it through the three layers built above gives this.

// After — hand a shared ledger and guard to every task, and persist line items
async function runDailyJobs(tasks: { id: string; model: string; run: Function }[]) {
  const ledger = new TokenLedger();
  const guard = new BudgetGuard(
    { runCeiling: 4.0, taskCeiling: 0.8, softRatio: 0.8 },
    ledger,
  );
  const results = await Promise.allSettled(
    tasks.map((t) =>
      t.run((model: string, invoke: any) =>
        guardedCall(guard, ledger, t.id, model, invoke),
      ),
    ),
  );
  await persistLedger(ledger.dump(), new Date());   // <- keeping line items pays off
  return results;
}

The diff is a handful of lines. Yet those few lines became the dividing line between "open the bill and go pale" operations and "grasp it every morning from the log" operations. You do not need to build a heavyweight monitoring stack; wrapping your existing parallel run in this thin layer is enough.

Allocating the numbers from real data

The amounts you put in runCeiling and taskCeiling are usually wrong if you guess. Too high and the guard never fires; too low and normal tasks get stopped. I decided mine after letting the ledger's line items accumulate for one to two weeks and looking at the actual distribution.

1. First, apply no cutoff at all and collect two weeks of TokenLedger dump().
2. List the per-run cost per task and look at the median and max. My link-audit task had a median of $0.12, stretching to $0.30 on occasion.
3. Set the per-task ceiling at 3–5x that max. I chose $0.8 — plenty of headroom for the normal range, while firmly stopping accident levels (which once topped $6).
4. Set the run-wide ceiling by taking the median of the total of all tasks in a night and multiplying by a safety factor of about 1.5. My night totaled around $2.5, so I set $4.0.

What is good about this method is that the basis lives in your own data. Rather than a general number, you derive it from the distribution your agents actually consume, so what should be stopped is stopped and what should pass is passed with high probability. In three months of operation, there were zero cases of mistakenly stopping a normal task.

Don't set the numbers once and forget them; review the line items monthly and adjust. The number of articles you handle and the model unit prices keep changing, so the ceilings need to be quietly kept in step.

Gotchas I hit in production

It took some painful lessons to get this working as designed. Worth recording.

First, missed metering on streaming responses. The confirmed token value only appears after the response completes, so even if a huge output runs during streaming, record is called last. That means the guard only takes effect "from the next call." A runaway within a single call slips through. As a countermeasure, the per-task ceiling needs to be set well above the expected consumption of one average call, yet below accident level. I aim for "3–5x the expected."

Second, double counting from retries. When you retry on a network error, the tokens of the failed call may actually be billed too. Forget to record the pre-retry consumption and the ledger diverges from the real bill. I fixed it to record on every retry. Every week I reconcile the billing dashboard against the dump() total to check the discrepancy stays within 5%. At first it was off by more than 20%, and most of that gap was missed retry records.

Third, overlooking the quality drop from downgrade. The output of a task downgraded to the fast model naturally drops in quality. When a downgrade fires during article generation, a thin article comes out. So for any run where a downgrade fired even once, I flag the output and route it to human review. Silently trading quality to protect cost is the entrance to a different kind of accident.

Fourth, concurrent writes to the shared ledger. Every sub-agent touches the same TokenLedger instance, so record is called concurrently. With Node's single thread a synchronous Map update does not race, but split off workers or separate processes and the story changes. For now I decided to run single-process and stated that premise in a comment on the ledger. If you make a change that breaks the premise, you need to switch to an atomic increment. Leaving the premise in words makes it harder for future-me to cause an accident half a year later.

These were all the kind of problem you cannot spot from a design diagram alone. They only surface after actually running one night in production. That is exactly why I recommend setting the ceilings quite low at first and deliberately tripping halt to verify the behavior.

What changed after adopting it

Since adding this governance layer, my monthly token cost returned to its pre-accident level, and more than that, the unease of "I can't predict what it will cost" is gone. Every night's run log now always carries run total: $X, so I notice anomalies the same day. The opening accident — learning about it only on next month's bill — has not happened once since.

To put a number on it: the token cost that swelled to about 3.2x in the accident month has stayed within ±10% of baseline in all three months since the guard went in. More than lowering the unit price through optimization, the reassurance of having a ceiling matters most to me.

The next step I'd recommend is to drop just the TokenLedger into your existing agents and collect a week of line items. Cutoff can come after. Just seeing in numbers what your agents actually spend, and on what, naturally reveals where to set the ceilings. I myself only realized, after I started keeping a ledger, that the task I assumed was the culprit was innocent — the plain, routine work was what added up.

I hope this gives you a foothold for your own design if you, too, are grappling with cost opacity.