There is a moment in production where your agent's response time quietly stretches. What used to come back in three seconds now takes twenty. The cause is probably a slow upstream API, or a single hop in your multi-agent flow that grew heavier over time. From the user's chair, though, it just looks broken. I have been shipping iPhone apps as a solo developer since 2014, and across that catalog (which has now passed 50 million cumulative downloads) I have watched this slow drift happen more times than I would like to admit. The hardest part is that nothing breaks. The agent runs, the tests pass, the response is even technically correct — it is only the experience around it that has quietly fallen out of shape.
Most articles on latency stop at "set a timeout and retry on failure." Production agents face something more nuanced every single turn. How many seconds do I really have left? Is this a paying customer whose request deserves more time? Should I return half an answer fast, or a full answer late? Should I skip the optional enrichment step entirely because the budget is almost gone? Hard-coding those judgments into the prompt is a losing battle. The model can be steered, but it cannot reliably read a clock that you do not give it. The agent itself needs to know what time it is, how much it has spent, and how much it has left.
I call this idea a Time Budget. Instead of treating timeout as an external constraint clamped onto the call, you treat it as a budget the agent consciously spends. There are three reasons this small reframe pays off. First, it converts an external mechanism (the timeout your platform enforces) into a first-class object that flows through your code, where you can read and reason about it. Second, it lets you express a single deadline that propagates through every layer of a multi-agent flow, rather than each layer guessing at its own clock. Third, it gives the agent a vocabulary for graceful behavior — "I am going to skip the optional step because I only have 800 milliseconds left" is a sentence the system can act on, instead of a guess buried inside a prompt.
What follows is the design I have been running on Antigravity agents for several months, the implementation pattern, and the production traps that are not obvious until you hit them. The code samples are TypeScript because that is how I tend to ship, but the same shapes translate cleanly to Python or Go — the structure of the object matters more than the language.
Why timeouts alone are not enough
Timeouts are not new. Every HTTP client takes one. Antigravity agent configs can specify an execution ceiling. So why bother with anything more?
Once you go to production, three situations break the "timeout-only" model.
The first is the multi-hop tool call where each layer only knows its own timeout. Suppose agent A calls agent B, and B calls an upstream API. A is configured to give up at 30 seconds. If B is independently configured at 45, it ignores A's clock entirely. When A times out, B keeps spinning on its own budget and ends up writing orphaned errors into your logs.
The second is treating high-priority and low-priority traffic the same. It is not reasonable to clamp a paying customer's request and a free-tier batch job at the same 20 seconds. But baking "you can take longer because this user pays" into the prompt is a smell — it warps the model's reasoning instead of speaking to the runtime.
The third is missing the moment to give up. If only three seconds remain and you are about to fire a tool call that you know averages four, starting it is wasted budget. An agent that does not know what time it is cannot make that call. To put numbers on it: in one of my apps, I traced a class of incidents where the agent burned an entire 18-second budget on a tool that historically took 11 seconds to return, only to error out and leave the user with nothing. After adding a canAfford check, those incidents simply stopped — the agent fell back to a faster path and the user got an answer, just a slightly less elaborate one.
A related symptom of "no clock" is the over-eager retry. Many agent frameworks retry tool calls automatically. Without a shared budget, retries can each take their full timeout, multiplying the latency the user actually sees. With a budget, you can wrap the retry loop so that each attempt is bounded by the remaining budget rather than the original tool timeout, which is a meaningfully different number ten seconds into a request.
The fix is a small mental flip. Instead of a timeout that the outside world enforces on the agent, give the agent a budget it spends. Once it knows the elapsed time, the remaining time, and the priority, it can adapt on its own. The rest of this article is the implementation of that flip.
The smallest possible Time Budget object
A useful Time Budget needs only three values:
- a start time (
startedAt) - a deadline (
deadline) - a priority
Bind them together and pass that one object through your agent code. In TypeScript, the minimum implementation is small enough to fit on screen.
// time-budget.ts
export type Priority = "critical" | "high" | "normal" | "low";
export class TimeBudget {
private readonly startedAt: number;
private readonly deadline: number;
readonly priority: Priority;
constructor(opts: { totalMs: number; priority?: Priority }) {
this.startedAt = Date.now();
this.deadline = this.startedAt + opts.totalMs;
this.priority = opts.priority ?? "normal";
}
/** Elapsed time in ms since this budget started. */
elapsed(): number {
return Date.now() - this.startedAt;
}
/** Remaining ms — clamped to 0 once we are past the deadline. */
remaining(): number {
return Math.max(0, this.deadline - Date.now());
}
/** True when the budget has run out. */
isExpired(): boolean {
return this.remaining() === 0;
}
/** Can we afford a task of this estimated duration, plus a safety margin? */
canAfford(estimatedMs: number, safetyMs = 200): boolean {
return this.remaining() >= estimatedMs + safetyMs;
}
/** Hand a fraction of the remaining time to a child task. */
spawn(ratio = 0.5, priority?: Priority): TimeBudget {
const childMs = Math.floor(this.remaining() * ratio);
return new TimeBudget({ totalMs: childMs, priority: priority ?? this.priority });
}
}The two methods that earn their keep here are canAfford and spawn. canAfford answers "can I start this without blowing the budget?" and is the first thing your agent checks before any tool call. spawn carves a slice of the remaining budget for a child task, so a parent with 20 seconds left can pass 10 down without losing track of the global deadline.
My grandfathers, both shrine carpenters, used to point out that you cannot pretend the lumber is infinite — the budget for the whole structure has to be drawn on paper before the first cut. A time budget feels like the same instinct, applied to runtime instead of timber.
Propagating the budget through tool calls
Antigravity agents are essentially compositions of tool calls. If you wrap each tool with something that takes a TimeBudget, the budget naturally rides along.
// budgeted-tool.ts
import { TimeBudget } from "./time-budget";
type ToolFn<I, O> = (input: I, signal?: AbortSignal) => Promise<O>;
export function withBudget<I, O>(
tool: ToolFn<I, O>,
estimatedMs: number,
fallback?: (input: I) => O,
): ToolFn<I & { budget: TimeBudget }, O> {
return async (input, signal) => {
const { budget, ...rest } = input;
if (!budget.canAfford(estimatedMs)) {
if (fallback) return fallback(input);
throw new Error(
`BudgetExceeded: required=${estimatedMs}ms remaining=${budget.remaining()}ms`,
);
}
// Wire the remaining budget into an AbortController so the call can be cut off.
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), budget.remaining());
try {
const merged = signal
? AbortSignal.any([signal, controller.signal])
: controller.signal;
return await tool(rest as I, merged);
} finally {
clearTimeout(timer);
}
};
}The wrapper does two jobs. First, it refuses to start a call it cannot afford, falling back to a degraded path if you supplied one. Second, it forces a hard cutoff once the budget is exhausted, by chaining an AbortSignal into the underlying tool. If your tool fans out into more downstream requests, they all get cancelled together.
In practice the fallback slot is where graceful degradation lives. A product search tool that normally returns 30 results from a vector index can fall back to a five-result keyword search when only a sliver of the budget remains.
An adaptive scheduler that listens to remaining time
With the budget in place, you can ask your planner to read remaining time and switch strategies. I call this an adaptive scheduler.
// adaptive-scheduler.ts
import { TimeBudget } from "./time-budget";
type Step = {
name: string;
estimatedMs: number;
importance: "must" | "should" | "nice";
run: (budget: TimeBudget) => Promise<unknown>;
};
export async function runWithinBudget(
steps: Step[],
budget: TimeBudget,
): Promise<Array<{ name: string; status: "ok" | "skipped" | "expired" }>> {
// 1) must first, then should, then nice — all within the same shared budget.
const ordered = [...steps].sort((a, b) => {
const w = { must: 0, should: 1, nice: 2 } as const;
return w[a.importance] - w[b.importance];
});
const results: Array<{ name: string; status: "ok" | "skipped" | "expired" }> = [];
for (const step of ordered) {
if (budget.isExpired()) {
results.push({ name: step.name, status: "expired" });
continue;
}
if (!budget.canAfford(step.estimatedMs)) {
// Only must-steps run when we cannot strictly afford them, with a clamped child budget.
if (step.importance === "must") {
const child = budget.spawn(0.95);
await step.run(child);
results.push({ name: step.name, status: "ok" });
} else {
results.push({ name: step.name, status: "skipped" });
}
continue;
}
const child = budget.spawn(step.estimatedMs / Math.max(budget.remaining(), 1));
await step.run(child);
results.push({ name: step.name, status: "ok" });
}
return results;
}The shape that earns its keep is the must / should / nice triage. For a customer support agent the pieces might be:
- must: classify the user's intent (cheap, but the response cannot work without it)
- should: pull related FAQ entries via vector search
- nice: surface analogous past tickets (heavy but flattering)
With 20 seconds available you do all three. With five, you run the must-step and tell the user "here is a quick answer" instead of stalling for the perfect reply. From the user's side, an honest fast answer beats an excellent late one almost every time.
Graceful cuts — degrading politely instead of erroring out
Throwing on timeout is the worst possible exit. The pattern I use in production is a "graceful cut": when the budget is about to expire, drop one tier in quality and ship what you have.
// graceful-cut.ts
export async function withGracefulCut<T>(
budget: TimeBudget,
primary: () => Promise<T>,
fallback: () => T | Promise<T>,
reservedMs = 500,
): Promise<{ value: T; degraded: boolean }> {
const primaryBudget = budget.spawn(
Math.max(0, budget.remaining() - reservedMs) / Math.max(budget.remaining(), 1),
);
try {
if (primaryBudget.remaining() <= 0) throw new Error("no time for primary");
const value = await primary();
return { value, degraded: false };
} catch {
const value = await fallback();
return { value, degraded: true };
}
}The non-obvious detail is reservedMs. You explicitly hold back a slice of the budget for the fallback to run. Without it, the primary path consumes everything, and when you fall through to the fallback you are already out of time.
Here is a story from one of my apps. A translation agent was supposed to fall back to a local cache whenever the upstream API timed out. In reality, the primary call ate the entire budget and the fallback never had a chance to run, so the user got a crash instead of a cached translation. The day I shipped the reservedMs reservation, the agent suddenly learned to give up gracefully — quietly, almost politely. I still remember being a little surprised at how much that small change changed the personality of the system.
Choosing the right initial budget for your agent
A question I get from other solo developers is "what number do I put in there for totalMs?" There is no universal answer, but there is a pattern that helped me reach reasonable numbers without guesswork.
Start by classifying the user-facing context. A chat interface where the user is actively waiting can usually tolerate around 8 to 12 seconds before users start to disengage. A background workflow (for example, "summarize my inbox once an hour") can tolerate 60 seconds or more. A real-time assistant that is interleaving with voice input is much tighter — you are looking at 1 to 3 seconds for any single agent step or you break the conversation rhythm.
Then look at your P90 latency for the request type. If P90 is currently 6 seconds, set your initial budget to roughly P90 plus a small safety margin — perhaps 7 to 8 seconds. The temptation is to set it generously high "just in case," but a too-generous budget means the agent never has reason to make the trade-offs the system is built for. The whole point of the design is to put the agent in a posture where it has to decide.
I usually start with a relatively tight budget and watch the graceful cut rate. If the cut rate is above 5%, the budget is too tight; if it is below 0.5%, the budget is too loose. The sweet spot in the products I run has been around 1 to 2% — a small but meaningful fraction of requests degrade gracefully, which means the system is exercising its fallback path often enough to keep that path healthy. Code paths that are never exercised tend to rot, even when they are kept short.
The shrine carpentry instinct returns here. There is no point sizing a structure for a load that will never come, and there is no point starving it for one that absolutely will. The right number is the one that matches what the structure is actually expected to carry, with a small but honest margin for surprise.
Integrating with Antigravity agent runners and the broader SDK
The pattern above lives mostly in your application code, but it works best when it shakes hands with the agent runtime. In Antigravity, an agent run is wrapped by the SDK's task executor, which already handles its own retries and timeouts. The trick is to let the runtime know about your TimeBudget so the two clocks do not fight.
The pattern I use is to pass the budget through as part of the agent's invocation context, and have a thin adapter that translates "remaining time" into the SDK's own timeout setting on every tool registration:
// antigravity-runner.ts
import { TimeBudget } from "./time-budget";
type RunOpts = { budget: TimeBudget; toolName: string };
export async function runWithBudget<T>(
invoke: (signal: AbortSignal, timeoutMs: number) => Promise<T>,
{ budget, toolName }: RunOpts,
): Promise<T> {
const remaining = budget.remaining();
if (remaining <= 0) throw new Error(`Budget already empty before ${toolName}`);
const ctrl = new AbortController();
const t = setTimeout(() => ctrl.abort(), remaining);
try {
return await invoke(ctrl.signal, remaining);
} finally {
clearTimeout(t);
}
}Two details matter here. The first is that timeoutMs is forwarded into the SDK call, so any retries the runtime does on its own are clamped by the same budget. The second is that the AbortController is shared, so an external cancel from the runtime (for example because the surrounding agent is itself out of time) cleanly cuts everything below it. If you skip these and just rely on your own timer, you get the worst of both worlds — the SDK retries until its own timeout, and the agent above has already moved on.
A subtle benefit is that this adapter unifies the place where you record telemetry. Whether the cancel came from you, from the SDK, or from a parent agent, you log it through the same path. When I look back at outage post-mortems, the ones that took longest to diagnose were the ones where the cancellation source was unclear. A single funnel removes that ambiguity.
If your agent uses streaming tools (for example, a tool that emits partial output to a websocket), the budget must be checked between chunks rather than only before the call. A small helper makes that obvious:
// streamed-tool.ts
import { TimeBudget } from "./time-budget";
export async function* boundedStream<T>(
source: AsyncIterable<T>,
budget: TimeBudget,
): AsyncGenerator<T> {
for await (const chunk of source) {
if (budget.isExpired()) break;
yield chunk;
}
}The streaming case is where I see the largest user-visible difference. Without budget-aware streaming, an agent that runs over time keeps sending chunks until the platform-level timeout kills the connection — which usually shows up as a jarring cut in the middle of a sentence. With budget-aware streaming, the agent finishes the current sentence, signals "ending here," and lets the orchestration layer hand back to the user gracefully.
Three traps I only spotted in production
Sketching the design is one thing. Running it in front of users is another. After about half a year of running budget-aware agents in production, three things bit me.
The first was forgetting to keep estimatedMs honest. You set "this tool averages 800ms" on day one. Six months later that upstream is averaging 1,500ms. Your canAfford keeps green-lighting calls that should be skipped, and your overshoot rate climbs. The fix I settled on is to compute estimatedMs dynamically from the last 24 hours of telemetry — using P90 latency as the estimate is a simple rule that holds up well in practice.
The second was splitting the budget evenly across child agents with spawn(ratio). If a parent fans out to three children and you give each one third, the last child runs after the others have already eaten their share — and effectively starts on an empty tank. You need to weight by importance, or hold back a meaningful slice for the final child. There was a stretch where my solo apps were generating over 1.5 million yen per month from AdMob, and during that window a recommendation flow built on a multi-agent fan-out started taking a noticeable extra second per response. The cause was exactly this even split.
The third was letting priority overrides stampede. "Paying users are critical" sounds reasonable until your paying users surge at lunchtime and free-tier users see catastrophic latency. I now use priority only to scale the time allocation, never to decide whether something runs at all. That guard alone removed a whole class of incident.
There is also a fourth that I almost missed because it manifests slowly: configuration drift. The constants you choose for safetyMs, reservedMs, and the various estimatedMs values feel obviously correct on day one. Six months later they are quietly out of sync with reality and nobody can remember the original reasoning. I now keep the budget constants in a single configuration module with a comment block explaining the chosen value, the metric it is based on, and the date of the last review. It feels like overkill until the day a new contributor (or future-me) needs to change one of them and is grateful that the rationale is right there.
Anti-patterns I tried and stepped away from
Before the design above settled, I tried a handful of variations that did not survive contact with production. Listing them seems more useful than describing only the version that worked, because if you are reading this looking for ideas, you might be tempted by the same dead ends.
The first was using a single global timeout instead of a budget object. It looks simpler — you just pass a deadline timestamp around. In practice, every layer ended up reaching for Date.now() and recalculating remaining time differently, and the lack of a shared method like canAfford meant inconsistent decisions across the stack. The object with methods looks heavier on paper but pays for itself within a week of use.
The second was tying budget directly to model selection. The thinking was: "fast model when budget is tight, smart model when there is time." This is correct in principle but became fragile when the smart model was slow because of upstream issues, not because of intrinsic latency. The budget should change behavior, not switch models, because model choice should be steered by capability fit and only secondarily by latency. I now keep model selection on a separate decision rule and let the budget govern step skipping and depth of search.
The third was making priority an integer (1–10). Granular numbers feel sophisticated but turn into a bikeshed — every code review ends up debating whether something is a 6 or a 7. Four named priorities (critical, high, normal, low) gives you the same expressive power without the room for argument, and the names make pull requests easier to read.
The fourth was sharing one budget object across an entire request lifecycle, including database calls and unrelated background work. That muddled the picture: a slow database query unrelated to the AI portion would burn the agent's apparent budget. The cleaner shape is to scope budgets to the agent invocation itself, and let other system layers carry their own timeouts. Separation of concerns is not just a code-organization principle — for budgets it actually reduces incident frequency, because you can pinpoint which subsystem is slow.
The fifth was hiding the budget from the agent's reasoning entirely, on the theory that the model should "just be fast." But once I started exposing remaining time as a structured input the agent could see (a single integer in a system message — "you have N milliseconds remaining"), the model became noticeably better at writing concise responses when time was short. Agents that know they are running late really do change their tone, and users seem to appreciate that more than I expected.
Observation, tuning, and the improvement loop
A budget design is only as good as your telemetry. I keep three signals and only those three.
The first is "budget utilization." Per request, log (elapsed / total) and chart the P50, P90, and P99. P99 hugging 1.0 means the budget is too tight; P90 stuck near 0.3 means you are leaving headroom on the table. Aiming for a P90 in the 0.6–0.8 zone has been a comfortable resting point in my own work.
The second is "step skip rate." For each should and nice step, what fraction of requests skip it? Anything skipping above 30% is no longer "nice" — it is "too heavy for the budget you actually have." Either rewrite that step or replace it with a lighter substitute.
The third is "graceful cut rate." How often did you fall through to the fallback path? When this number jumps, either your upstream is degrading or your planner has bloated. Alerting on a sudden change here lets you spot the slowdown before users send you a "feels rougher than usual" message.
This whole observation loop is a sibling of the work in Designing AI Agent Error Recovery — Patterns for Pipelines That Refuse to Stop. Once you stop thinking about retry and timeout as separate things and start thinking about "how do we apportion the limited resource called time," the design tightens up considerably.
If you also want to explore cost in the same frame, Antigravity Agent Cost Optimization — Halving Token Spend with Design Decisions treats tokens and time as twin budgets. And for validation before flipping the switch in production, the techniques in Antigravity Agent Shadow Mode — A Production Rollout Strategy for Trying New Versions Safely translate well to budget verification — you can shadow your budget config the same way you shadow a model upgrade.
Where to start tomorrow
Once your agent starts paying attention to remaining time, the operator's anxiety drops in a quiet way. Instead of users telling you "feels slower lately," the agent itself can say "I will reply with a short summary this time." The acknowledgment moves from your inbox into the system.
If you want a low-risk first move, pick the single agent in your stack with the most unstable latency, and add a TimeBudget only there. Resist the urge to roll the pattern out everywhere at once. I waited about two months between the first integration and a wider rollout, and that slower path is what gave me a feel for how time flows through my own product. I think that detour is what made the budget numbers I eventually settled on actually fit my service, instead of just inheriting numbers from someone else's blog post.
If any of this resonates with a problem you are sitting on, I would be glad if it helped even a little.