"The agent stopped working" — I looked into the report and found it had hit a temporary rate limit on an external API and just... stopped. No retry, no fallback, nothing. That experience made me rethink error recovery design for agents from the ground up.
In a user-facing application, "An error occurred, please try again" is acceptable. But AI agents are supposed to run autonomously. If they can't recover from errors on their own, you've lost half the value of using agents in the first place.
This article categorizes the typical failure patterns in AI agent pipelines and walks through recovery strategies with implementation code for each.
Three Failure Patterns Unique to Agents
Unlike standard web application error handling, AI agents have their own distinct failure modes.
Pattern 1: Hallucination loops. The agent misinterprets tool call results and repeatedly attempts to access resources that don't exist. For example, misreading a file listing as "there are 1000 files" and then trying to access non-existent file paths over and over.
Pattern 2: Tool chain failures. In a multi-step pipeline where Step A's output feeds into Step B, a failure in the middle causes everything downstream to collapse. Simple retries won't help.
Pattern 3: Context overflow. Long-running agents hit the context window limit and "forget" initial instructions or critical intermediate results. The agent drifts off-task but keeps running.
Let's look at defenses for each.
Smart Retry with Exponential Backoff
Simple retries aren't enough. Rate limits call for exponential backoff, and hallucination loops require the judgment to stop retrying altogether.
interface RetryConfig {
maxRetries: number;
baseDelayMs: number;
maxDelayMs: number;
retryableErrors: Set<string>;
// Threshold for early termination when the same error repeats
sameErrorThreshold: number;
}
const DEFAULT_RETRY_CONFIG: RetryConfig = {
maxRetries: 5,
baseDelayMs: 1000,
maxDelayMs: 30000,
retryableErrors: new Set([
"rate_limit_exceeded",
"server_error",
"timeout",
"connection_reset",
]),
sameErrorThreshold: 3,
};
async function withSmartRetry<T>(
fn: () => Promise<T>,
config: RetryConfig = DEFAULT_RETRY_CONFIG
): Promise<T> {
let lastErrorCode = "";
let sameErrorCount = 0;
for (let attempt = 0; attempt <= config.maxRetries; attempt++) {
try {
return await fn();
} catch (error: unknown) {
const errorCode = (error as { code?: string }).code ?? "unknown";
// Non-retryable errors are thrown immediately
if (\!config.retryableErrors.has(errorCode)) {
throw error;
}
// Detect consecutive identical errors (hallucination loop signal)
if (errorCode === lastErrorCode) {
sameErrorCount++;
if (sameErrorCount >= config.sameErrorThreshold) {
throw new Error(
`Same error ${sameErrorCount} times in a row: ${errorCode} (possible hallucination loop)`
);
}
} else {
sameErrorCount = 1;
lastErrorCode = errorCode;
}
if (attempt === config.maxRetries) throw error;
// Exponential backoff with jitter
const delay = Math.min(
config.baseDelayMs * Math.pow(2, attempt) + Math.random() * 500,
config.maxDelayMs
);
await new Promise((r) => setTimeout(r, delay));
}
}
throw new Error("unreachable");
}The key is sameErrorThreshold. When the same error code occurs three times in a row, it's not a problem that retrying will solve — it's a sign the agent is repeating the same flawed decision. Cutting off early is the first line of defense against hallucination loops.
Fallback Chains to Prevent Cascade Failures
In multi-step pipelines, each step should have alternative approaches (fallbacks) ready.
type StepFn<TIn, TOut> = (input: TIn) => Promise<TOut>;
interface PipelineStep<TIn, TOut> {
name: string;
primary: StepFn<TIn, TOut>;
fallbacks: StepFn<TIn, TOut>[];
validate: (output: TOut) => boolean;
}
async function executeStep<TIn, TOut>(
step: PipelineStep<TIn, TOut>,
input: TIn
): Promise<{ output: TOut; usedFallback: boolean; method: string }> {
// Try primary
try {
const output = await withSmartRetry(() => step.primary(input));
if (step.validate(output)) {
return { output, usedFallback: false, method: "primary" };
}
} catch {
// Fall through to fallbacks
}
// Try fallbacks in order
for (let i = 0; i < step.fallbacks.length; i++) {
try {
const output = await withSmartRetry(() => step.fallbacks[i](input));
if (step.validate(output)) {
return { output, usedFallback: true, method: `fallback_${i}` };
}
} catch {
continue;
}
}
throw new Error(`All methods failed for step "${step.name}"`);
}
// Example: Web page content fetch step
const fetchContentStep: PipelineStep<string, string> = {
name: "fetch_content",
primary: async (url) => {
const res = await fetch(url);
return res.text();
},
fallbacks: [
async (url) => {
// Try cache
const cached = await cacheStore.get(url);
if (\!cached) throw new Error("Cache miss");
return cached;
},
async (url) => {
// Degrade gracefully with metadata
return `[Content fetch failed] URL: ${url} — proceeding without summary`;
},
],
validate: (content) => content.length > 0,
};Notice the third fallback. Even when content fetching completely fails, it passes meta-information ("fetch failed") and keeps the pipeline moving. The agent receives this and continues processing with whatever other data it has. The pragmatic tradeoff of "imperfect results beat a stopped pipeline" is essential in agent pipeline design.
Circuit Breakers to Contain Failure Propagation
When an external service is down for an extended period, continuing to retry wastes resources and causes timeout cascades that slow the entire pipeline.
enum CircuitState {
CLOSED = "closed", // Normal (requests pass through)
OPEN = "open", // Tripped (errors returned immediately)
HALF_OPEN = "half_open" // Probing (let 1-2 requests through)
}
class CircuitBreaker {
private state: CircuitState = CircuitState.CLOSED;
private failureCount = 0;
private lastFailureTime = 0;
private successCount = 0;
constructor(
private readonly name: string,
private readonly failureThreshold: number = 5,
private readonly recoveryTimeMs: number = 60_000,
private readonly halfOpenSuccessThreshold: number = 2
) {}
async execute<T>(fn: () => Promise<T>): Promise<T> {
if (this.state === CircuitState.OPEN) {
if (Date.now() - this.lastFailureTime > this.recoveryTimeMs) {
this.state = CircuitState.HALF_OPEN;
this.successCount = 0;
} else {
throw new Error(`CircuitBreaker [${this.name}]: OPEN — requests blocked`);
}
}
try {
const result = await fn();
this.onSuccess();
return result;
} catch (error) {
this.onFailure();
throw error;
}
}
private onSuccess(): void {
if (this.state === CircuitState.HALF_OPEN) {
this.successCount++;
if (this.successCount >= this.halfOpenSuccessThreshold) {
this.state = CircuitState.CLOSED;
this.failureCount = 0;
}
} else {
this.failureCount = 0;
}
}
private onFailure(): void {
this.failureCount++;
this.lastFailureTime = Date.now();
if (this.failureCount >= this.failureThreshold) {
this.state = CircuitState.OPEN;
}
}
}
// Create per-service circuit breakers
const openaiBreaker = new CircuitBreaker("openai_api", 3, 30_000);
const searchBreaker = new CircuitBreaker("search_api", 5, 60_000);The circuit breaker has three states. In normal operation it's CLOSED (requests pass through), when failures exceed the threshold it goes OPEN (errors returned immediately), and after a cooldown period it transitions to HALF_OPEN (letting 1-2 test requests through).
For agent pipelines, the best practice is to create separate circuit breakers for each external service. If the search API goes down, everything else — file operations, code execution — continues working normally.
Fault Injection Testing for Resilience Verification
Once you've implemented recovery mechanisms, you need to verify they actually work. Apply chaos engineering principles to agent testing.
class FaultInjector {
private rules: Map<string, { probability: number; errorType: string }> = new Map();
addRule(toolName: string, probability: number, errorType: string): void {
this.rules.set(toolName, { probability, errorType });
}
shouldInjectFault(toolName: string): { inject: boolean; errorType?: string } {
const rule = this.rules.get(toolName);
if (\!rule) return { inject: false };
return Math.random() < rule.probability
? { inject: true, errorType: rule.errorType }
: { inject: false };
}
}
// Test example
async function testPipelineResilience(): Promise<void> {
const injector = new FaultInjector();
// Simulate search API timing out 50% of the time
injector.addRule("web_search", 0.5, "timeout");
// Simulate file writes failing 20% of the time
injector.addRule("file_write", 0.2, "disk_full");
const results = { succeeded: 0, failed: 0, usedFallback: 0 };
for (let i = 0; i < 100; i++) {
try {
const result = await runPipelineWithInjector(injector);
results.succeeded++;
if (result.usedFallback) results.usedFallback++;
} catch {
results.failed++;
}
}
console.log(`Succeeded: ${results.succeeded}/100`);
console.log(`Used fallback: ${results.usedFallback}`);
console.log(`Complete failures: ${results.failed}`);
// Require 90%+ success rate
if (results.succeeded < 90) {
throw new Error("Resilience test failed: success rate below 90%");
}
}This test verifies that over 90% of 100 pipeline runs succeed. Even under harsh conditions where the search API fails half the time, properly functioning fallbacks and retries keep the overall pipeline success rate above 90%.
After introducing this test, we caught bugs like "the fallback's validate function was always returning false" and "the circuit breaker's recoveryTime was too short, transitioning to HALF_OPEN before the API had actually recovered."
Agents don't stop in production because errors happen. They stop because error recovery fails. Combining the three patterns from this article — smart retry, fallback chains, and circuit breakers — creates a design where individual failures don't cascade into full pipeline shutdowns. As a concrete next step, identify the most frequent error in your agent pipeline and add one fallback for it.
This article is published in full as a free sample of our premium content. Premium articles on Antigravity Lab cover production-ready implementation patterns like these, with working code examples. To access all premium articles, consider joining our membership.