Protecting Your Agent Stack's Known-Good State with a Single Lockfile — Change-Budget Design for an Era of Simultaneously Moving Parts

The change budget — move only one axis at a time

A change budget is the number of axes you allow yourself to move in a single update, measured from the last known-good state. I set it to 1 as a rule.

The reasoning is simple. If you only ever move one axis at a time, then when a regression appears the suspect is always narrowed to one. You do not even need to bisect. You just revert the axis you moved last.

You confirm this mechanically, right before an unattended run. A preflight reads the current environment, counts the axes that differ from the known-good state, and stops the run if the budget is exceeded.

#!/usr/bin/env python3
# preflight.py — run this right before an unattended job; halt if the change budget is exceeded
import subprocess
import sys
import yaml
 
 
def read_current_axes() -> dict:
    """Read each axis's version from the current stack. Axes that can't be read become 'unknown'."""
    def sh(cmd: str) -> str:
        try:
            out = subprocess.run(
                cmd, shell=True, capture_output=True, text=True, timeout=20
            )
            return out.stdout.strip() or "unknown"
        except Exception:
            return "unknown"
 
    return {
        "antigravity_ide": sh("agy --ide-version 2>/dev/null"),
        "antigravity_cli": sh("agy --version 2>/dev/null | head -1"),
        "default_model": sh("agy config get default_model 2>/dev/null"),
        "node": sh("node --version 2>/dev/null | tr -d 'v'"),
    }
 
 
def diff_axes(known: dict, current: dict) -> list:
    """Return the axes that disagree between known-good and current."""
    drifted = []
    for axis, good_value in known.items():
        now = current.get(axis, "unknown")
        if now != good_value:
            drifted.append((axis, good_value, now))
    return drifted
 
 
def main() -> int:
    with open("stack.known-good.yml", encoding="utf-8") as f:
        manifest = yaml.safe_load(f)
 
    budget = manifest.get("change_budget", 1)
    known = manifest["known_good"]["axes"]
    current = read_current_axes()
    drifted = diff_axes(known, current)
 
    if not drifted:
        print("preflight: matches known-good. Allowing the unattended run.")
        return 0
 
    print(f"preflight: {len(drifted)} axis/axes drifted from known-good (budget {budget})")
    for axis, good, now in drifted:
        print(f"  - {axis}: {good} -> {now}")
 
    if len(drifted) > budget:
        print("preflight: change budget exceeded. Aborting the unattended run.")
        print("  -> Verify axes one at a time, then update known-good once confirmed.")
        return 1
 
    print("preflight: within budget. Allowing the run, but recording this axis as today's suspect.")
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

The goal of this script is not to prevent regressions. It is to keep the suspect list down to one when a regression does happen. You move exactly one axis within budget, run, and if the result is good you update known-good and move to the next. If it breaks, reverting that one axis takes you back to the known-good state.

Retrieval commands like agy --ide-version may be named differently in your version. Axes that cannot be read are treated as unknown and counted as drifted. Rather than optimistically assuming an unreadable axis is "probably the same," it is safer to put it explicitly onto the suspect list.

The bisection procedure when a regression appears

If you keep to the budget, bisection is unnecessary. But there are moments when several axes unavoidably move at once — a migration, for instance. July's CLI migration was exactly that. For those moments, decide the isolation procedure in advance.

The key is the "smallest reproducing case" in step 3. If you judge by running all four sites, each trial takes too long, and the bisection will not finish in a practical number of rounds. I keep a separate reproducing case that takes about thirty seconds — "generate a single article and look at whether the tables break." Isolating the cause and running the heavy production job are best kept as separate concerns.

To make the revert reliable, note the pinning method for each axis too. The IDE build is an explicit stable version, the CLI is a pinned version, the model is explicitly set in config, and dependencies are the lockfile. If there is an axis you "cannot revert," the bisection breaks down there — so whether a revert path exists is precisely what you should check beforehand.

Wiring it into unattended runs — let the preflight stop it

Put the preflight at the very start of the scheduled run. If the budget is exceeded, exit without running the main job, and notify only on failure. In unattended operation, "silently producing broken output" is the worst outcome; "we skipped today because the budget was exceeded" is far easier to handle.

#!/usr/bin/env bash
# nightly.sh — the entrance to an unattended pipeline
set -euo pipefail
 
if ! python3 preflight.py; then
  echo "Skipped today's unattended run because the change budget was exceeded." >&2
  # Notify on failure only (silent when healthy)
  exit 0
fi
 
# Run the main job only when the preflight passes
python3 generate_articles.py

As a result, the state of an unattended run resolves into three cases.

The "needs review" row is the biggest payoff of this design. As long as you kept to the budget, even when something goes wrong the suspect is narrowed to the single axis you moved last. The starting point for the investigation becomes a log, not a hunch.

What running it taught me

Before I put this foundation in place, diagnosing a regression often burned half a day. Too many axes had moved, so all I could do was revert them one by one and try again.

After narrowing the budget to one and updating known-good every time, diagnosis became mostly just "revert the axis you moved last." In my own records, isolation time dropped from the order of tens of minutes to the order of a few minutes. There is nothing flashy about it, but the sense of calm each morning changed.

There is a cost, though: updates get slower. Because you can only move one axis at a time, catching up on all four takes several days, with a good-state confirmation between each step. It sits honestly at odds with the urge to try a new feature right away.

So I keep the experimental environment separate from the production unattended one. In the experimental environment I try the latest without worrying about budget, and if it is good I promote it into production "one axis at a time." Exploring the moving parts and keeping revenue-critical unattended operation stable are simply different goals.

Another lesson was to only put "revertible axes" on the budget. An axis with no revert path breaks the premise of bisection. Keeping things in a form you can reliably revert — version pins and lockfiles — is what supports the whole idea of a change budget.

In an era where the moving parts shift all at once, you cannot freeze everything. What you can do is order the changes one at a time, and always leave a road back to the last known-good state. It is quiet preparation, but the wider you let agents run unattended, the more the presence or absence of this foundation begins to tell.

I hope this helps with your own setup. Thank you for reading.