Designing Antigravity Agents That Survive Multi-Hour Batches — A 4-Layer Durability Pattern From the Trenches

Layer 1: Three Checkpoint Granularities, Combined

The first move is to stop thinking about checkpoints as a single concept. I split them into task-, step-, and token-level checkpoints, and combine them.

Step 1: Task-level checkpoints

A task is a unit a human would name aloud, like "review AdMob settings for these 30 wallpaper apps." One task is one JSON file, written atomically.

# checkpoint_task.py
import json, time, os
from pathlib import Path
 
CHECKPOINT_DIR = Path(".checkpoints/tasks")
CHECKPOINT_DIR.mkdir(parents=True, exist_ok=True)
 
def save_task(task_id: str, status: str, payload: dict) -> None:
    """Write a task-level checkpoint atomically."""
    record = {
        "task_id": task_id,
        "status": status,           # pending / running / completed / failed
        "updated_at": time.time(),
        "payload": payload,
    }
    target = CHECKPOINT_DIR / f"{task_id}.json"
    tmp = target.with_suffix(".tmp")
    tmp.write_text(json.dumps(record, ensure_ascii=False, indent=2))
    os.replace(tmp, target)         # atomic rename

The atomic os.replace is the whole point. A process killed mid-write never leaves a half-written file. It is unglamorous, but the durability of a long batch is the sum of many small "joints that don't break when something falls" — and this is one of them. I think about that a lot when I write infrastructure code.

Step 2: Step-level checkpoints

A step is a sub-unit inside one task. For an app it might be: fetch → analyze → propose → validate → apply → confirm, six steps. When a task crashes, restarting at step six instead of step one is the difference between a 30-second recovery and a 30-minute one.

In production I store step-level state in SQLite. The reason will become clear shortly.

Step 3: Token-level checkpoints

The most granular layer is the agent's internal state — its message history, prior summaries, and tool-call traces. I snapshot this every 8,000 tokens, not every step. The point is to have a stable anchor before context bloat starts to corrode quality.

"Three layers sounds like a lot" is a fair objection, but the alternative is worse. Task-only checkpoints restart all six steps; step-only ones let context balloon. Splitting by granularity is the path of least suffering, in my experience.

Layer 2: Picking the Persistence Backend — SQLite / KV / Durable Objects / R2

Where you save matters more than people expect. Here is my honest comparison of the four backends I have lived with.

The recommendation I give myself: use SQLite during development, and a hybrid of KV plus Durable Objects in production. Task-level state (change history for each of the apps I run) lives in KV. Step-level work that demands strict "apply once" semantics (mutating AdMob settings) lives in a Durable Object. R2 is where compressed context snapshots from Day 1 of a batch end up, in case I want to replay them weeks later.

Here is the SQLite step store I actually use.

# step_store.py
import sqlite3, json, time
from pathlib import Path
 
DB = Path(".checkpoints/steps.sqlite")
DB.parent.mkdir(parents=True, exist_ok=True)
 
def init_db() -> None:
    with sqlite3.connect(DB) as c:
        c.execute("""
            CREATE TABLE IF NOT EXISTS steps (
                task_id TEXT NOT NULL,
                step_idx INTEGER NOT NULL,
                status TEXT NOT NULL,
                payload TEXT NOT NULL,
                updated_at REAL NOT NULL,
                PRIMARY KEY (task_id, step_idx)
            )
        """)
 
def upsert_step(task_id: str, step_idx: int, status: str, payload: dict) -> None:
    with sqlite3.connect(DB) as c:
        c.execute(
            "INSERT OR REPLACE INTO steps VALUES (?, ?, ?, ?, ?)",
            (task_id, step_idx, status, json.dumps(payload, ensure_ascii=False), time.time()),
        )
 
def next_step(task_id: str) -> int:
    """Return the next step index to execute; completed ones are skipped."""
    with sqlite3.connect(DB) as c:
        row = c.execute(
            "SELECT MAX(step_idx) FROM steps WHERE task_id = ? AND status = 'completed'",
            (task_id,),
        ).fetchone()
    return (row[0] + 1) if row and row[0] is not None else 0

INSERT OR REPLACE makes the write idempotent and MAX(step_idx) makes restart trivial. Because SQLite is a single file, you can sync it to Dropbox or iCloud Drive and resume on a different machine. Mine lives in Dropbox: a batch I started on a MacBook one morning routinely resumes on the iMac in the afternoon. The first time I saw it happen, the value of pushing state out to external storage stopped being abstract.

Layer 3: Idempotent Restore — Never Run a Side Effect Twice

Even with perfect checkpoints, a careless restore can double-apply effects. I have personally lived through: AdMob campaigns created twice, Stripe charges fired twice, alert emails delivered twice. All of them were avoidable with a single discipline — give every side-effecting call an operation key, and check the KV before invoking it.

# idempotent_call.py
import hashlib, json
from typing import Callable, Any
 
def idempotent_call(
    operation_key: str,
    fn: Callable[[], Any],
    kv_get: Callable[[str], str | None],
    kv_put: Callable[[str, str], None],
) -> Any:
    """Make a side-effectful call idempotent.
 
    operation_key: e.g. "admob/app123/campaign/2026-05-24"
    fn:            the actual side-effectful call
    kv_get/kv_put: KV accessors
    """
    cached = kv_get(operation_key)
    if cached is not None:
        return json.loads(cached)         # second time: no-op, return cached result
 
    result = fn()
    kv_put(operation_key, json.dumps(result, ensure_ascii=False))
    return result
 
def make_key(*parts: str) -> str:
    """Stable key with hashed disambiguation."""
    raw = "/".join(parts)
    return hashlib.sha1(raw.encode("utf-8")).hexdigest()[:16] + "_" + parts[0]

The race between kv_get → fn() → kv_put is real. In production I wrap that block in blockConcurrencyWhile on a Durable Object for the truly critical operations, and use a KV write with nx-style semantics for the eventually-consistent ones. I recommend reserving Durable Objects for the few places where strict ordering is non-negotiable; KV is fine almost everywhere else.

A side benefit: those operation keys double as an audit log. I can query KV at month-end and see exactly which AdMob settings changed, on which day, and what the agent decided. That alone has saved me hours of revenue forensics.

Layer 4: Summarization — Compress the Past Every 8,000 Tokens

The quietest failure mode in long batches is context-quality drift. In my setup, the agent's output noticeably degraded past 40,000 tokens of accumulated context, and around 80,000 it would start to loop. The cause is not that the agent "remembers too much" — it is that it is dragged into yesterday's irrelevant detail and can no longer make a new decision cleanly.

My summarization rule is simple. Every time the context crosses 8,000 tokens, compress everything except the last 2,000 tokens into a "progress summary," then drop the head. Drop here means "remove from the active context"; the head is still archived in R2 in case I want to audit later.

# context_summarizer.py
from dataclasses import dataclass
from typing import Callable
 
@dataclass
class Message:
    role: str       # user / assistant / tool
    content: str
    tokens: int
 
def summarize_context(
    messages: list[Message],
    summarize_fn: Callable[[list[Message]], str],
    threshold: int = 8000,
    keep_tail_tokens: int = 2000,
) -> list[Message]:
    total = sum(m.tokens for m in messages)
    if total < threshold:
        return messages
 
    # Keep the last keep_tail_tokens worth of messages
    tail: list[Message] = []
    used = 0
    for m in reversed(messages):
        if used + m.tokens > keep_tail_tokens:
            break
        tail.insert(0, m)
        used += m.tokens
 
    head = messages[: len(messages) - len(tail)]
    summary_text = summarize_fn(head)
    summary_msg = Message(role="assistant", content=f"[summary] {summary_text}", tokens=len(summary_text) // 2)
    return [summary_msg] + tail

Concretely: before I added summarization, a 12-hour batch averaged about $48. With summarization on the same workload, it dropped to about $14 — roughly a 70% reduction. Output quality did not regress; if anything, the agent stayed more grounded because the summary forced "what are we even doing right now" to be made explicit at every compaction. Context bloat is as much a clarity problem as a cost problem.

Observability — Four Numbers That Tell You the Layers Are Working

Designing the layers is not enough; you need to know they are functioning. These are the four metrics I track daily.

• Restart success rate: when I deliberately SIGKILL the process, the percentage of restarts that resume from the last successful step. Target: 99%.
• Redundant execution rate: same operation_key calling fn() more than once, divided by total operation_key count. Target: under 0.1%.
• Context compression ratio: pre-compression token total divided by post-compression total. Currently ~4.2× for me.
• Median batch wall-clock time: median completion time for a fixed workload. The baseline against which every change is measured.

A nightly cron aggregates these and posts to my Slack. Without these numbers I would never have caught the time I accidentally set a 24-hour TTL on the KV — restart success rate had quietly drifted to 70% over a weekend, and feel alone would not have flagged it for months.

Real Costs From a 24-Hour Batch

Once the system runs, having a real cost model makes decisions easier. Here are actual numbers from a 24-hour batch that reviewed AdMob configuration across the wallpaper apps I run.

• Input tokens: ~18M (with summarization)
• Output tokens: ~2.5M
• Tool calls: ~6,400 (external APIs: ~1,200)
• Estimated total: ~$14 (without summarization: ~$47)
• Storage: KV + Durable Objects + R2, ~$0.55 per month

About $14 for a full day, roughly $420 per month if I ran one every day. Against AdMob revenue from the portfolio, that pays for itself many times over — but the moment I removed summarization and idempotency, cost tripled and incident rate spiked. The durability layers are doing double duty: reliability and cost control are not separate concerns here.

The Pragmatic Decisions I've Actually Settled On

Three rules I use as defaults when scoping a new batch agent.

1. Under 30 minutes? Skip three of the four layers. Task-level checkpoints are enough. Add idempotency only around the few side-effecting calls.
2. Over two hours? Always add summarization. Cost savings are large and the agent's judgement holds up better. I tend to flag any task that might breach 60 minutes.
3. Over eight hours? Move the critical-path writes onto Durable Objects. KV's eventual consistency hides race conditions that only surface in long horizons. Strong consistency is worth its modest complexity tax at that scale.

You do not have to land here on day one. I did not. The four layers grew in over about six months of watching things break and adding only the next layer that the data demanded. Durability design, for me, assembles itself in the reverse of how you'd expect: not from a complete blueprint drawn up front, but from reinforcing one broken joint at a time until the structure stands on its own.

What I Want to Try Next

I am not done. Cross-agent checkpoint sharing — letting a Day 1 agent hand its compressed state to a Day 2 agent through Antigravity's Manager Surface — is still rough, and I am redesigning the handoff protocol. I also want an objective metric for summarization quality: whether a compressed history reproduces the same decisions the full history would have produced.

Long-running batch agents have quietly moved my indie app business and adjacent media operations to a scale I could not have reached by hand. I am still learning, and I would be glad if any of this helps someone else build the next layer. Thanks for reading.