Tracking AI Agent Decisions in Antigravity: Implementing Decision Logs and Explainability

Making Tool Selection Transparent — Before/After

The biggest payoff from decision log design comes from side-effectful tools. Changing the tool definition alone is often enough to make agent reasoning visible.

Before — no reason field, no traceability

# ❌ Bad: tool has no reason field
tools_bad = [
    {
        "name": "write_file",
        "description": "Write content to a file",
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {"type": "string"},
                "content": {"type": "string"}
            },
            "required": ["path", "content"]
        }
    }
]
# → Can't tell from outside why this specific file was chosen
# → Can reconstruct what happened, but not why
# → Can't determine whether to fix the prompt or the tool definition

After — reason and scope fields required

# ✅ Good: reason and scope fields required, intention made explicit
tools_good = [
    {
        "name": "write_file",
        "description": "Write content to a file. Has side effects — reason and scope required",
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {"type": "string"},
                "content": {"type": "string"},
                "reason": {
                    "type": "string",
                    "description": "Explain why this file is being modified (50+ characters)"
                },
                "scope": {
                    "type": "string",
                    "enum": ["specific_file", "pattern_match"],
                    "description": "Change scope (specific_file: only this file)"
                }
            },
            "required": ["path", "content", "reason", "scope"]
        }
    }
]
# ✅ What the log captures:
#   - Which file was changed (path)
#   - Why that file was selected (reason field)
#   - Intended scope of change (scope field)
#   - Confidence score estimated from reason richness

The file overwrite incident I described earlier would have been preventable with the scope field. Requiring the agent to select scope="specific_file" makes the intended change boundary explicit — and when something goes wrong, the scope choice is right there in the log.

The important nuance: this design isn't constraining the agent's behavior. It's providing a place to articulate what the agent was already thinking. The agent still decides freely, but its reasoning becomes part of the record.

Structured Reasoning Logs with ReAct

To capture "what was considered" rather than just "what was done," we can apply the ReAct pattern (Reasoning + Acting) to record the alternatives and factors that influenced each decision.

from dataclasses import dataclass, field
import re
from typing import Optional
 
@dataclass
class ReasoningRecord:
    """Records a structured reasoning step in the ReAct pattern"""
    step: int
    thought: str
    observation: str
    action_chosen: str
    alternatives_considered: list[str] = field(default_factory=list)
    decision_factors: list[str] = field(default_factory=list)
    uncertainty_flags: list[str] = field(default_factory=list)
 
    def to_dict(self) -> dict:
        return {
            "step": self.step,
            "thought": self.thought,
            "observation": self.observation,
            "action_chosen": self.action_chosen,
            "alternatives": self.alternatives_considered,
            "factors": self.decision_factors,
            "uncertainties": self.uncertainty_flags
        }
 
def build_structured_logging_prompt() -> str:
    """Build a system prompt that elicits structured reasoning logs"""
    return """For each step, think through the following structure:
 
<reasoning>
Current situation: [what you observe]
Alternatives considered: [option A, option B, option C]
Reason for choice: [why this action was selected]
Uncertainty flags: [anything you're not sure about]
</reasoning>
<action>
[actual tool call or response]
</action>
 
Before any side-effectful action (write, delete, external API), explicitly note
your uncertainty flags. When uncertain, choose the most conservative option."""
 
def parse_reasoning_from_response(response_text: str, step: int) -> Optional[ReasoningRecord]:
    """Extract structured reasoning from agent response"""
    reasoning_match = re.search(r'<reasoning>(.*?)</reasoning>', response_text, re.DOTALL)
    action_match = re.search(r'<action>(.*?)</action>', response_text, re.DOTALL)
 
    if not reasoning_match:
        return None
 
    reasoning_text = reasoning_match.group(1).strip()
    action_text = action_match.group(1).strip() if action_match else "(no action recorded)"
 
    uncertainties = re.findall(r'Uncertainty flags?:\s*(.+)', reasoning_text)
    alternatives = re.findall(r'Alternatives considered:\s*(.+)', reasoning_text)
    factors = re.findall(r'Reason for choice:\s*(.+)', reasoning_text)
    observation = re.search(r'Current situation:\s*(.+)', reasoning_text)
 
    return ReasoningRecord(
        step=step,
        thought=reasoning_text,
        observation=observation.group(1) if observation else "",
        action_chosen=action_text[:100],
        alternatives_considered=alternatives,
        decision_factors=factors,
        uncertainty_flags=uncertainties
    )

The most valuable element here is the uncertainty flag. Asking the agent to explicitly record what it's unsure about surfaces exactly the places where prompt design needs work. In my experience, debugging time dropped significantly once I could say "step 4 had uncertainty flags about file scope" rather than starting from scratch every time something went wrong.

Quality Scoring and the Improvement Cycle

Once logs are accumulating, the next step is automating the analysis. Three metrics drive the quality score:

Low Confidence Rate — the proportion of steps where confidence is "low." Above 20% suggests the task specification or tool descriptions need refinement.

Uncertainty Density — how many steps have uncertainty flags. High density in specific domains (file operations, external APIs) signals a gap in the context or guidelines you're providing the agent.

Tool Sequence Consistency — how consistent the sequence of tool calls is across the same task type. High variance usually reflects prompt ambiguity.

def analyze_decision_logs(log_entries: list[dict]) -> dict:
    """Analyze decision logs and produce dashboard metrics"""
 
    total = len(log_entries)
    if total == 0:
        return {"error": "No log entries found"}
 
    confidence_counts = {"high": 0, "medium": 0, "low": 0}
    for entry in log_entries:
        c = entry.get("confidence", "medium")
        confidence_counts[c] = confidence_counts.get(c, 0) + 1
 
    low_confidence_rate = confidence_counts["low"] / total
 
    tool_sequence = [
        e["tool"]["name"]
        for e in log_entries
        if e.get("tool")
    ]
 
    uncertainty_steps = sum(
        1 for e in log_entries
        if e.get("uncertainty_flags") and len(e["uncertainty_flags"]) > 0
    )
    uncertainty_density = uncertainty_steps / total
 
    quality_score = max(0, min(100, int(
        100
        - (low_confidence_rate * 40)
        - (uncertainty_density * 20)
    )))
 
    return {
        "quality_score": quality_score,
        "total_steps": total,
        "confidence_distribution": confidence_counts,
        "low_confidence_rate": f"{low_confidence_rate:.1%}",
        "uncertainty_density": f"{uncertainty_density:.1%}",
        "tool_sequence": tool_sequence,
        "recommendations": generate_recommendations(
            low_confidence_rate,
            uncertainty_density
        )
    }
 
def generate_recommendations(
    low_confidence_rate: float,
    uncertainty_density: float
) -> list[str]:
    recs = []
 
    if low_confidence_rate > 0.20:
        recs.append(
            "Low confidence rate exceeds 20%. "
            "Consider making task instructions more specific or enriching tool descriptions."
        )
 
    if uncertainty_density > 0.30:
        recs.append(
            "High uncertainty flag density. "
            "Consider providing more context or adding decision guidelines "
            "to the system prompt for uncertain scenarios."
        )
 
    if not recs:
        recs.append("Quality metrics are healthy. Use this prompt design as a reference for other tasks.")
 
    return recs

Running this improvement cycle for six months raised the average quality score for my agents from 62 to 81. The number isn't the whole story — but there was a palpable shift from "the agent kind of works" to "the agent behaves according to its design."

For multi-agent systems (see the Multi-Agent Orchestration Implementation Guide), assign a distinct session_id to each agent and merge logs for analysis. This lets you immediately identify which agent in the chain caused a quality issue.

Where to Store Decision Logs — A Two-Layer JSONL + SQLite Setup

After implementing DecisionLogger, the first problem I ran into was an unglamorous one: where to put the logs.

Design articles rarely talk about the storage layer, but choosing poorly means your carefully recorded logs pile up unanalyzed. For my first month, I wrote one JSON file per task. Once the directory passed 400 files, I stopped opening any of them — a predictable outcome I should have seen coming.

I have since settled on a two-layer setup: writes go to date-based JSONL files (one entry per line), and analysis happens in SQLite.

The reason for JSONL on the write side is that I never want the logging mechanism itself to crash an agent mid-run because of a schema issue. A logger that stops the agent defeats its own purpose. Appending a single line to an open file almost never fails.

import json
import sqlite3
from datetime import datetime
from pathlib import Path
 
LOG_DIR = Path("logs/decisions")
 
def append_entry(entry: dict) -> None:
    """During execution, just append to JSONL. Validate on the read side."""
    LOG_DIR.mkdir(parents=True, exist_ok=True)
    path = LOG_DIR / f"{datetime.now():%Y-%m-%d}.jsonl"
    with path.open("a", encoding="utf-8") as f:
        f.write(json.dumps(entry, ensure_ascii=False) + "\n")
 
def load_into_sqlite(db_path: str = "decisions.db") -> int:
    """Import JSONL into SQLite before analysis. Skip corrupt lines."""
    conn = sqlite3.connect(db_path)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS decisions (
            session_id TEXT, step INTEGER, tool TEXT,
            confidence TEXT, reason TEXT, ts TEXT,
            PRIMARY KEY (session_id, step)
        )
    """)
    imported = 0
    for jsonl in sorted(LOG_DIR.glob("*.jsonl")):
        for line in jsonl.read_text(encoding="utf-8").splitlines():
            try:
                e = json.loads(line)
                conn.execute(
                    "INSERT OR IGNORE INTO decisions VALUES (?,?,?,?,?,?)",
                    (e["session_id"], e["step"],
                     (e.get("tool") or {}).get("name"),
                     e.get("confidence"), e.get("reason"), e.get("timestamp")),
                )
                imported += 1
            except (json.JSONDecodeError, KeyError):
                continue  # never let one bad line block the whole import
        
    conn.commit()
    conn.close()
    return imported

The detail that matters here is that the read side silently skips corrupt lines. A half-written line from an agent that died mid-run is a real occurrence, and if your import pipeline halts on it, you will eventually stop looking at logs altogether.

Some volume numbers for perspective: in my environment a task averages 30–50 steps, and each entry is roughly 1KB. Even running several tasks daily, that stays well under a few hundred megabytes per month — comfortably within SQLite territory for a solo developer. A warehouse like BigQuery only starts earning its complexity once a team needs to aggregate months of logs across people. Not building big infrastructure on day one is itself a design decision that keeps the practice sustainable.

One caution: the reason field absorbs fragments of filenames and user input the agent saw. If you ever ship logs to an external analytics service, add a masking step at import time for strings shaped like email addresses or API keys. Data that is harmless in a local SQLite file changes meaning the moment it changes location.

Common Pitfalls

Verbosity affects performance — requiring structured reasoning output on every step increases token usage and latency. My approach: detailed logging during development, summary logging in production, with a flag to switch between modes. For long-running agents like Antigravity Background Agents, log only side-effectful steps in detail.

Reason fields get filled in generically — making reason required doesn't prevent the agent from writing "following user instructions." Character count checks catch blank fields, but not generic ones. Specificity estimation helps here:

def estimate_reason_specificity(reason: str) -> str:
    """Estimate confidence level from reason text specificity"""
    import re
    specificity_markers = [
        r'\b[a-zA-Z_]+\.(py|ts|json|md|txt)\b',         # file names
        r'\b(function|class|def|const|let|var)\s+\w+',   # code elements
        r'\b(because|since|given that|due to)\b',         # causal expressions
        r'\d+',                                            # numeric values
    ]
    matched = sum(1 for p in specificity_markers if re.search(p, reason, re.IGNORECASE))
 
    if matched >= 3:
        return "high"
    elif matched >= 1:
        return "medium"
    else:
        return "low"

Logs grow faster than insight — the real challenge in long-term operation is extracting useful patterns from accumulated logs. My rule: weekly, collect sessions where quality_score fell below 70. Monthly, classify problems into three types: wrong tool selection, scope expansion, unclear reasoning. Quarterly, fix the most common pattern and A/B test the revised prompt against the original.

For systems running autonomously around the clock — like the one described in Running Autonomous AI Agents 24/7 with Trigger.dev — decision logs serve as an audit trail that enables post-hoc verification of any agent judgment call.

Deciding How Much to Delegate — From Gut Feeling to Logged Evidence

After six months of running decision logs, the biggest payoff was not faster incident investigation. It was being able to draw the line of "how much do I let the agent do on its own" from data instead of intuition.

Developers adopting agent-based tools often tell me they have no framework for deciding what to delegate. I was the same: "this feels risky, I'll review it by hand" or "this seems fine, let it run." Once logs accumulate, you can redraw that line per task type, with numbers.

The three tiers I use:

• Full delegation: across the last 20 sessions of that task type, low_confidence_rate is under 10% and there are zero uncertainty flags on tool calls with side effects. The agent runs without human review
• After-the-fact review: low_confidence_rate between 10–25%. Execution isn't blocked, but I review a digest of only the side-effect steps afterwards
• Prior approval: above 25%, or any irreversible operation such as file deletion or external transmission. The agent must present a plan and wait for sign-off

The thresholds themselves carry no universal truth. What matters is that they came out of my own operational logs.

A concrete example: in my app business, I delegate drafting store listing copy and localizing release notes. Listing copy generation held a single-digit low_confidence_rate for weeks, so I moved it to full delegation. The act of publishing to the store, however, stays at prior approval — even though the logs look nearly spotless. An irreversible operation against a live product is something I've decided not to delegate, no matter how good the numbers get.

Logs give you grounds for delegation; whether you should delegate remains a business judgment. Learning to separate those two questions was the most valuable thing six months of logs taught me.

The line also doesn't stay drawn forever. Numbers shift with every model update. In any week where Antigravity ships a model change, I demote fully delegated tasks one tier, watch 20 sessions' worth of metrics, and only then restore them. It's unglamorous maintenance, but it's the most reliable defense I've found against "the behavior changed and nobody noticed."

What "Explainable AI" Actually Means

Let me say something slightly philosophical.

Ever since I started programming, I've felt that writing code is a form of thinking — that the act of making something forces clarity about why you're making it. In any craft where you work with your hands, the people who can put "why" into words are the ones who can think from the root when something goes wrong.

When we delegate judgment to AI agents, what are we releasing?

Explainability design is a technical requirement, but it's also an act of building trust between humans and AI systems. Using a system you can't explain is like following a convention whose origin no one can remember. The longer it goes on, the harder it becomes to trace the problem when something breaks.

At the same time, whether AI-generated "explanations" actually reflect genuine understanding is still an open question. When an agent writes "I chose this file because the task description mentioned updating the configuration," does that carry the same weight as a human explanation?

For now, I prioritize traceability over accuracy of explanation. Even if I can't fully understand every decision, a record I can inspect later lets me build trust incrementally. That's what logs are for.

The sense that technology exists to link people across distances still runs through my approach to agent design. Building something that operates autonomously is an act of investing trust in what you can't see. Decision logs are the foundation that makes that trust possible.

Where to Start — Add One reason Field Today

Decision log design doesn't require a large system. The minimum viable step is adding one reason field to the tool your agent uses most frequently.

Watch what reasons the agent writes. Observe the patterns. You'll start to see where your prompts are ambiguous and where your tool definitions have gaps. Fix one thing at a time.

That incremental process is what turns an agent that "kind of works" into an agent that behaves according to its design — and that users can rely on.

I'm still working through this myself, and I hope this is useful for others facing the same challenge.