Engineering Quality Into AI Agents — When Autonomous Execution Breaks and How to Prevent It
Autonomous AI agents degrade over time. This article shows how to catch the decay before it breaks, with multi-stage verification gates and a failure dictionary that lets agents self-recover — drawn from running four sites with multiple agents in parallel.
When I first started using AI agents, my biggest misconception was that "if it works once, it keeps working."
It doesn't. Agents change over time — not because the agent itself changes, but because everything around it does. External APIs update. Tool versions shift. Data formats evolve. And suddenly, without touching a single line of configuration, the behavior you depended on is gone.
As an indie developer, I run four AI tech blog sites (Dolice Labs) with multiple autonomous agents in parallel. This article draws on that to explain why quality decays, and how to design systems that stay reliable — with the actual code I run in production. The point is not to stare at a monitoring dashboard, but to build resilience into the design up front.
Four Patterns Where Autonomous Execution Breaks
Understanding failure modes is the prerequisite for preventing them. In practice, almost everything comes down to four patterns.
Pattern 1: Environment Drift
The tools and APIs your agent depends on change quietly. Git options change. API response shapes evolve. Auth tokens expire. The agent is identical, but the gap between its assumptions and reality accumulates until it fails abruptly. In my operation this happens once or twice a month — most recently, a hardcoded session path rotated and every file write failed at once.
Pattern 2: Prompt Drift
Your agent's instructions (a SKILL.md file, a system prompt) are a living document. Every maintenance pass shifts them slightly. Over time, contradictions accumulate. A classic case: the prose still says "twice a week" while the schedule was changed to "once a week." Unless you decide which one is canonical every time, the agent trusts the stale text.
Pattern 3: Data Corruption
The data your agent references — article lists, config files, execution logs — accumulates inconsistencies over time. I once had a reference-data path change (an underscore prefix appeared) while cat swallowed the error and logged "success" with empty output. A silent failure is far more dangerous than a loud one.
Pattern 4: Success Fixation
This is the hardest pattern to catch. When an approach worked in the past, agents tend to repeat it even as conditions change. "Last time this slug recovered fine" becomes reusing a slug you'd deliberately removed with a 410 — an SEO self-inflicted wound.
Catching Decay Early — Four Signals to Watch
Decay rarely arrives "all at once." There are usually precursors. Here are the signals I check daily, with thresholds. The absolute numbers depend on your scale; watch the day-over-day change rate rather than the raw value.
Signal
Meaning
Alert line (my setup)
Consecutive skips
Same task skipping ("no candidate") in a row
3 in a row → alert (likely structural)
Empty-output rate
Reference-data cat returning empty but proceeding
Even one empty → must log the gap
Gate failure rate
Share rejected by pre-push gates
Over 20%/week → revise the prompt
JA/EN count delta
Divergence between language counts
Delta ≥ 1 → halt (breeds 404s)
"Consecutive skips" gets underrated. When an agent reports "no suitable candidate" three times running, genuine exhaustion is rare; it's usually a structural cause like an unmounted directory or stale data. Treat skips as a normal branch and your pipeline can sit silently stalled for weeks.
✦
Thank you for reading this far.
Continue Reading
What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.
WHAT YOU'LL LEARN
✦The four decay patterns that break autonomous execution, plus the observable signals and thresholds that warn you before it fails
✦Idempotency, multi-stage verification gates, and a failure dictionary — the implementation and code for agents that self-recover
✦How quality assurance changes under Antigravity 2.0's multi-agent orchestration, from real parallel-operation experience
Secure payment via Stripe · Cancel anytime
✦
Unlock This Article
Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.
Monitoring to detect problems is not enough. Quality should be guaranteed by design — built into the structure of how your agents operate.
Principle 1: Make Agents Self-Report
At the end of every task, have the agent record what it did, what failed, and what it was uncertain about. Pin the format down — free-form logs drift in granularity every run.
# Fixed execution-log format (always appended at task end)LOG="${WS}/_updated_article_log/antigravitylab/$(TZ=Asia/Tokyo date +%Y-%m-%d)_upgrade.txt"{ echo "## $(TZ=Asia/Tokyo date '+%Y-%m-%d %H:%M JST')" echo "- target: $SLUG" echo "- success: rewrite, premium promotion, push" echo "- uncertain: third highlight leans abstract" echo "- skip reason (if any): none"} >> "$LOG"
Always stamp the date with TZ=Asia/Tokyo. Plain date is UTC, and a late-night run once overwrote the previous day's log file.
Principle 2: Design for Idempotency
Autonomous execution requires retries. When something fails and reruns, the rerun must not cause side effects.
# Idempotent write: unique temp name → verify → promoteTMP="$HOME/upg_${SLUG}_$(date +%s).mdx" # never a fixed name (avoids stale leftovers)build_article > "$TMP"grep -q "premium: true" "$TMP" || { echo "build failed"; exit 1; }mv "$TMP" "$JA"
A fixed filename silently mixes in leftovers from a write that died midway. Use a unique name containing the slug and a timestamp, and promote to production only when verification passes.
Principle 3: Embed Verification Steps — in Multiple Stages
After every significant operation, verify the output. Not one check, but several independent ones. I run four gates in series before every push.
The key: never batch the gates and the push into the same shell call. Write gate && git push on one line and the push fires before anyone reviews the gate output. I once pushed a violating article that way, because a warning scrolled past inside a single combined call. Confirm the gate result, then push in a separate call.
Principle 4: Remember Failures as a "Dictionary"
When an agent fails, store reusable knowledge, not just an error line. I record every failure as three fields.
Field
What to record
Symptom
What failed and how (observed facts only)
Root cause
Why it happened (confirmed, not hypothesized)
Permanent fix
How to prevent it next time (code or a check)
Once this failure dictionary passed roughly 100 entries, falling into the same hole twice became noticeably rare. Error logs scroll away; a dictionary persists as the rationale behind design decisions. Keep it in your agent's memory or an external file and reference it at the start of every task.
Three Interventions That Actually Worked
Enough theory. Three changes that had outsized impact in practice.
Intervention 1: Explicit Prohibition Lists
Describe what the agent must never do. Negative constraints are often more reliable than positive instructions.
## 🚫 Absolute Prohibitions
- Running npm install (fills VM disk, stops bash)
- Using AskUserQuestion (halts scheduled execution)
- Hardcoding session paths (breaks on rotation)
- Linking to articles that don't exist (causes 404s)
- Batching gates and git push into one call
Intervention 2: Dynamic Detection Over Hardcoding
# ❌ Hardcoded (breaks when session name changes)WS="/sessions/old-session-name/mnt/Dolice Labs"# ✅ Dynamic detection (always finds the right path)WS="$(ls -d /sessions/*/mnt/Dolice\ Labs 2>/dev/null | head -1)"
This alone eliminated an entire category of "it worked last time" failures.
Intervention 3: Verify Against the Remote, Not the Mirror
Your workspace mirror can be stale. Run pruning decisions and count checks against a fresh git clone of the remote, not the local copy. Trusting a local config and deleting an article that should have stayed is the worst kind of accident — and pinning the source of truth to the remote prevents it.
How Quality Assurance Changes in the Antigravity 2.0 Multi-Agent Era
Antigravity 2.0 has grown into a platform spanning desktop, CLI, SDK, and a Managed Agents API, with parallel orchestration of multiple agents at its core. On June 18, Gemini CLI folded into the Antigravity CLI, making the migration of automation a real, present task.
When you move from a single agent to managing a fleet of agents, the center of gravity for quality assurance shifts. The old problem was "one agent decays over time." The new headline problem is "multiple agents touch the same file at once and corrupt it."
The first thing I made work in parallel operation was separation at the file boundary. Use worktrees or branches to physically partition what each agent can touch, so two agents never write to the same article directory simultaneously. Cut tasks into "a unit I can review in 15 minutes," and write the three-line acceptance criteria before handing it over. The more parallelism you add, the smaller each task's granularity should become.
As background scheduled runs multiply, "consecutive skips" and "silent failures" get harder to see. In a multi-agent setup it matters even more that every agent writes a self-report log, and that counts and gate-failure rates are aggregated across agents. The more agents you run, the more the human role shifts from implementation to designing the observation points.
Three Steps to Start Today
You don't need all of this at once. In order of cost-effectiveness:
Add a one-line JA/EN count check (or your domain's consistency check) before push. Cheapest, prevents the most accidents.
Fix an execution-log format and always record success, failure, and skip reasons. You can't improve what you can't observe.
Create a single failure-dictionary file and add one entry — symptom, root cause, permanent fix — each time you get burned. Six months in, your operational baseline is a different thing.
Quality Assurance Is a Practice, Not a Checklist
Agent quality assurance is not a one-time setup. When something fails, understand why and record it. When something works unexpectedly well, name the pattern. Revisit your skill definitions regularly and prune what no longer applies.
Do this consistently for six months, and what you end up with is an agent that's hard to break and capable of self-recovery. You can't build a perfect agent from day one. But you can grow one, deliberately, over time. If you're an indie developer running several projects solo as well, I hope this gives your design a foothold.
Share
Thank You for Reading
Antigravity Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.