Capping Parallel Agents With a Token Budget — Designing a Guard That Stops Runaway Cost

Allocating a budget per task

Allocation first. I put "run" at the top level, with multiple "tasks" hanging beneath it. The key is to hold a ceiling for the whole run and a separate ceiling for each task. A run-only ceiling cannot stop a single runaway; a task-only ceiling lets the total go unbounded as parallelism grows.

// budget.ts — budget allocation definitions
export type Money = number; // USD. Everything internal is unified in USD
 
export interface BudgetPolicy {
  runCeiling: Money;        // upper limit for this whole run
  taskCeiling: Money;       // upper limit a single task may use
  softRatio: number;        // start degrading past this fraction (e.g. 0.8)
}
 
// Per-model unit price (per 1M tokens, USD). Keep in sync with real billing
export const PRICE_TABLE: Record<string, { input: Money; output: Money }> = {
  "fast":     { input: 0.30, output: 1.50 },
  "balanced": { input: 3.00, output: 15.0 },
  "deep":     { input: 8.00, output: 40.0 },
};
 
export function costOf(model: string, inTok: number, outTok: number): Money {
  const p = PRICE_TABLE[model] ?? PRICE_TABLE["balanced"];
  return (inTok / 1_000_000) * p.input + (outTok / 1_000_000) * p.output;
}

Always externalize the price table. Model prices get revised, and the model you use differs by site and by purpose. In my case article generation uses balanced, simple work like link audits uses fast, and only design review uses deep — I vary the allocation by the nature of the task.

There is also a reason everything is unified in USD internally. I tend to think in yen by site, but model prices are usually quoted in dollars, and if conversion formulas scatter across the codebase, you will get a digit wrong somewhere. In my first implementation I held some values in yen and some in dollars, and the ceiling comparisons became meaningless. The unit of the values you compare should be pinned in one place.

Accumulating real spend in real time

Next is metering, and this was the crux. The essence of the accident was "I could only learn consumption after the fact," so I keep a ledger that holds the running total while the job is live.

// ledger.ts — a ledger that accumulates token spend in real time during a run
import { costOf, Money } from "./budget";
 
interface Entry {
  taskId: string;
  model: string;
  inTok: number;
  outTok: number;
  cost: Money;
  at: number;
}
 
export class TokenLedger {
  private entries: Entry[] = [];
  private byTask = new Map<string, Money>();
  private runTotal: Money = 0;
 
  record(taskId: string, model: string, inTok: number, outTok: number): Money {
    const cost = costOf(model, inTok, outTok);
    this.entries.push({ taskId, model, inTok, outTok, cost, at: Date.now() });
    this.byTask.set(taskId, (this.byTask.get(taskId) ?? 0) + cost);
    this.runTotal += cost;
    return cost;
  }
 
  taskSpend(taskId: string): Money {
    return this.byTask.get(taskId) ?? 0;
  }
 
  total(): Money {
    return this.runTotal;
  }
 
  // emit line items for later reconciliation
  dump(): Entry[] {
    return [...this.entries];
  }
}

The important part is to run record immediately after every LLM call. Even running in parallel, as long as the ledger instance is shared, runTotal sums consumption across all agents. At first I split the ledger per agent and could not see the whole-run total — a silly implementation. "Collect values you want to sum in one place" sounds obvious, but it surprisingly breaks once you go parallel.

The need for real time may seem excessive at first. But with ten running in parallel, no one can predict when the sum will hit the ceiling. If one task suddenly gets heavy, the whole total jumps even if the others are light. After-the-fact aggregation only gives you the number after the jump. Because you accumulate while running, the judgment "we are now at 80% of the ceiling overall" holds, and you can act while there is still room to stop. I feel this single point — seeing the running total while in flight — is the greatest value of the governance layer.

Always save the dump() line items after a run. The hardest part of the opening accident was not knowing which task was the culprit. With line items kept, I can pinpoint the cause in five minutes the next morning. It pays off as insurance.

A hard cap with staged degradation

The third layer is cutoff. With allocation (budget) and metering (ledger) in place, I implement a BudgetGuard that reconciles the two and makes a call. The call has three stages.

Stage one is normal: do nothing until the soft threshold (e.g. 80% of the ceiling). Stage two is downgrade: once past the soft threshold, automatically switch subsequent calls to a cheaper model. Finish quality drops a little, but in many situations that beats stopping. Stage three is halt: at the hard cap, allow no further calls for that task (or the whole run).

// guard.ts — reconcile budget and ledger, decide on degradation
import { BudgetPolicy } from "./budget";
import { TokenLedger } from "./ledger";
 
export type Decision =
  | { action: "proceed"; model: string }
  | { action: "downgrade"; model: string }
  | { action: "halt"; reason: string };
 
export class BudgetGuard {
  constructor(
    private policy: BudgetPolicy,
    private ledger: TokenLedger,
  ) {}
 
  // run before every call; follow the returned action
  check(taskId: string, requestedModel: string): Decision {
    const run = this.ledger.total();
    const task = this.ledger.taskSpend(taskId);
 
    // hard cap: halt if either the whole run or the task exceeds
    if (run >= this.policy.runCeiling) {
      return { action: "halt", reason: `run ceiling $${this.policy.runCeiling} reached` };
    }
    if (task >= this.policy.taskCeiling) {
      return { action: "halt", reason: `task ${taskId} ceiling reached` };
    }
 
    // soft threshold: downgrade once run spend passes softRatio
    if (run >= this.policy.runCeiling * this.policy.softRatio) {
      return { action: "downgrade", model: "fast" };
    }
 
    return { action: "proceed", model: requestedModel };
  }
}

Run check before a call. record is after. Keeping this before/after symmetry is the trick — forget either and budget leaks. I folded "guard, then call, then record" into a single function so agents cannot touch the pieces individually.

// guarded-call.ts — the sole entry point that always pairs check with record
import { BudgetGuard } from "./guard";
import { TokenLedger } from "./ledger";
 
export async function guardedCall(
  guard: BudgetGuard,
  ledger: TokenLedger,
  taskId: string,
  model: string,
  invoke: (model: string) => Promise<{ inTok: number; outTok: number; text: string }>,
): Promise<{ text: string; halted: boolean }> {
  const decision = guard.check(taskId, model);
  if (decision.action === "halt") {
    console.warn(`[budget] halt ${taskId}: ${decision.reason}`);
    return { text: "", halted: true };
  }
  const useModel = decision.action === "downgrade" ? decision.model : model;
  const res = await invoke(useModel);
  ledger.record(taskId, useModel, res.inTok, res.outTok);
  return { text: res.text, halted: false };
}

Now, however the agent is implemented, as long as the path to the LLM goes through guardedCall, the budget always applies.

Conversely, leave even one bypass and budget leaks through it. When I review agent code, the one thing I always check is that no raw call equivalent to invoke is written outside guardedCall. Narrowing the entry point to one is unglamorous, but it is the single most effective rule for turning governance from "operating carefully" into "operating with structural protection."

This staging may look excessive. But I once tried a halt-only design and got burned. A task like article generation does most of its lookup up front and finishes at the end, so 80% of spend lands in the first half. If it touches the threshold and halts immediately in the first half, it gets cut off in a half-finished state, wasting the night's tokens entirely. Inserting downgrade as a middle stage lets you choose "finish with a cheaper model, but finish." The output is not perfect, but it beats zero. Protecting cost and not zeroing out the result are two goals worth thinking about separately.

Wiring it into Antigravity's parallel agents

Now to fold the three layers into Antigravity's parallel agents. Antigravity lets you run multiple sub-agents at once, but each calls the LLM independently, so left alone the combined consumption is invisible. So I create the ledger and guard exactly once at run scope and hand the same instance to every sub-agent.

// orchestrate.ts — distribute a shared guard to parallel agents
import { TokenLedger } from "./ledger";
import { BudgetGuard } from "./guard";
import { guardedCall } from "./guarded-call";
 
async function runDailyJobs(tasks: { id: string; model: string; run: Function }[]) {
  const ledger = new TokenLedger();
  const guard = new BudgetGuard(
    { runCeiling: 4.0, taskCeiling: 0.8, softRatio: 0.8 },
    ledger,
  );
 
  // the point: every task shares the same ledger / guard
  const results = await Promise.allSettled(
    tasks.map((t) =>
      t.run((model: string, invoke: any) =>
        guardedCall(guard, ledger, t.id, model, invoke),
      ),
    ),
  );
 
  // persist line items after the run (cause analysis takes 5 minutes)
  await persistLedger(ledger.dump(), new Date());
  console.log(`[budget] run total: $${ledger.total().toFixed(3)}`);
  return results;
}

Antigravity's parallel agents leave a work log and artifacts per sub-agent. I write the ledger's run total, and whether a downgrade or halt fired, into those. With that in place, opening the artifact the next morning shows the night's cost and how many times the guard kicked in at a glance. The agent's output and the cost record sit in the same place, so I can review quality and cost together rather than separately — which I like.

Using Promise.allSettled here is deliberate. With Promise.all, when one task fails on a halt, it would drag down and discard the results of the other healthy tasks. Even if a task stopped on budget overrun, I want the rest to finish and collect their results, so allSettled is the practical choice.

I put the run-scope runCeiling at $4.0 and the per-task ceiling at $0.8 to match one night's job size for my six-site operation. I derived them from the billing delta of the first accident. Adjust them to your own scale.

A naive parallel run vs. one routed through the guard

Let me put the before and after side by side in code. Back when I was having accidents, the naive implementation looked roughly like this.

// Before — no guard. Consumption is only knowable from the post-run bill
async function runDailyJobs(tasks: { id: string; run: Function }[]) {
  // each task hits the LLM however it likes; no sum, no ceiling
  return Promise.all(tasks.map((t) => t.run()));
}

At a glance there is nothing wrong with it, and most days it runs quietly. But with no ceiling anywhere, a single runaway is unstoppable, and you only notice the anomaly once the bill settles.

Routing it through the three layers built above gives this.

// After — hand a shared ledger and guard to every task, and persist line items
async function runDailyJobs(tasks: { id: string; model: string; run: Function }[]) {
  const ledger = new TokenLedger();
  const guard = new BudgetGuard(
    { runCeiling: 4.0, taskCeiling: 0.8, softRatio: 0.8 },
    ledger,
  );
  const results = await Promise.allSettled(
    tasks.map((t) =>
      t.run((model: string, invoke: any) =>
        guardedCall(guard, ledger, t.id, model, invoke),
      ),
    ),
  );
  await persistLedger(ledger.dump(), new Date());   // <- keeping line items pays off
  return results;
}

The diff is a handful of lines. Yet those few lines became the dividing line between "open the bill and go pale" operations and "grasp it every morning from the log" operations. You do not need to build a heavyweight monitoring stack; wrapping your existing parallel run in this thin layer is enough.

Allocating the numbers from real data

The amounts you put in runCeiling and taskCeiling are usually wrong if you guess. Too high and the guard never fires; too low and normal tasks get stopped. I decided mine after letting the ledger's line items accumulate for one to two weeks and looking at the actual distribution.

1. First, apply no cutoff at all and collect two weeks of TokenLedger dump().
2. List the per-run cost per task and look at the median and max. My article-generation task had a median of $0.12, stretching to $0.30 on occasion.
3. Set the per-task ceiling at 3–5x that max. I chose $0.8 — plenty of headroom for the normal range, while firmly stopping accident levels (which once topped $6).
4. Set the run-wide ceiling by taking the median of the total of all tasks in a night and multiplying by a safety factor of about 1.5. My night totaled around $2.5, so I set $4.0.

What is good about this method is that the basis lives in your own data. Rather than a general number, you derive it from the distribution your agents actually consume, so what should be stopped is stopped and what should pass is passed with high probability. In three months of operation, there were zero cases of mistakenly stopping a normal task.

Don't set the numbers once and forget them; review the line items monthly and adjust. The number of articles you handle and the model unit prices keep changing, so the ceilings need to be quietly kept in step.

Gotchas I hit in production

It took some painful lessons to get this working as designed. Worth recording.

First, missed metering on streaming responses. The confirmed token value only appears after the response completes, so even if a huge output runs during streaming, record is called last. That means the guard only takes effect "from the next call." A runaway within a single call slips through. As a countermeasure, the per-task ceiling needs to be set well above the expected consumption of one average call, yet below accident level. I aim for "3–5x the expected."

Second, double counting from retries. When you retry on a network error, the tokens of the failed call may actually be billed too. Forget to record the pre-retry consumption and the ledger diverges from the real bill. I fixed it to record on every retry. Every week I reconcile the billing dashboard against the dump() total to check the discrepancy stays within 5%. At first it was off by more than 20%, and most of that gap was missed retry records.

Third, overlooking the quality drop from downgrade. The output of a task downgraded to the fast model naturally drops in quality. When a downgrade fires during article generation, a thin article comes out. So for any run where a downgrade fired even once, I flag the output and route it to human review. Silently trading quality to protect cost is the entrance to a different kind of accident.

Fourth, concurrent writes to the shared ledger. Every sub-agent touches the same TokenLedger instance, so record is called concurrently. With Node's single thread a synchronous Map update does not race, but split off workers or separate processes and the story changes. For now I decided to run single-process and stated that premise in a comment on the ledger. If you make a change that breaks the premise, you need to switch to an atomic increment. Leaving the premise in words makes it harder for future-me to cause an accident half a year later.

These were all the kind of problem you cannot spot from a design diagram alone. They only surface after actually running one night in production. That is exactly why I recommend setting the ceilings quite low at first and deliberately tripping halt to verify the behavior.

What changed after adopting it

Since adding this governance layer, my monthly token cost returned to its pre-accident level, and more than that, the unease of "I can't predict what it will cost" is gone. Every night's run log now always carries run total: $X, so I notice anomalies the same day. The opening accident — learning about it only on next month's bill — has not happened once since.

To put a number on it: the token cost that swelled to about 3.2x in the accident month has stayed within ±10% of baseline in all three months since the guard went in. More than lowering the unit price through optimization, the reassurance of having a ceiling matters most to me.

The next step I'd recommend is to drop just the TokenLedger into your existing agents and collect a week of line items. Cutoff can come after. Just seeing in numbers what your agents actually spend, and on what, naturally reveals where to set the ceilings. I myself only realized, after I started keeping a ledger, that the task I assumed was the culprit was innocent — the plain, routine work was what added up.

I hope this gives you a foothold for your own design if you, too, are grappling with cost opacity.