Snapshot Your Agents' Effective Scope So You Notice the Day a Default Changes

I ran the agent that builds my delivery-size image derivatives the morning after applying the September 3rd update. Back came the usual derivatives, plus a draft of store listing copy I had never asked for.

I had not touched the definition file since the day before. What changed was not my side but the default. From 1.1.25 onward, a custom agent defined in Markdown inherits the surrounding skills, rules, and subagents unless you say otherwise.

For a long time I scoped my agents by not writing things down. The store-copy agent got a translation glossary and nothing else; the image agent got nothing at all — if I never handed it over, it could not see it. That assumption came apart cleanly on a morning when one default moved.

The narrower I want an agent to be, the more explicitly I write it down instead of leaning on a default. That is the line I hold now.

A Setting You Never Wrote Is Borrowed, Not Owned

Anything you leave unwritten is on loan from a default value. Borrowed things change when the lender decides they should. I am not complaining about that — I had simply forgotten I was borrowing.

The awkward part is that this kind of change raises no error. The agent sees more, so it does more. It still runs, which leaves you with nothing but a vague sense that the output got bigger than usual. As an indie developer juggling several apps and sites at once, that vague sense gets lost in the noise of a busy week.

So I stopped relying on the feeling and started relying on a diff. The idea is plain: before and after applying an update, write down what each agent can actually see, in a form a machine can compare.

Write the Effective Scope Into a Single JSON File

The script reads each definition file's front matter, reads the surrounding skills, rules, and subagents, and reconstructs the rule "add them if inheritance is on, leave them out if it is off." Here is what I use.

#!/usr/bin/env python3
"""agent_scope_snapshot.py — dump the effective scope of Markdown-defined agents.
 
  python3 agent_scope_snapshot.py --inherit-default off <config_dir> > before.json
  python3 agent_scope_snapshot.py --inherit-default on  <config_dir> > after.json
  python3 agent_scope_snapshot.py --diff before.json after.json
"""
import argparse, json, sys
from pathlib import Path
 
POOLS = ("skills", "rules", "subagents")
 
 
def read_front_matter(path):
    lines = path.read_text(encoding="utf-8").splitlines()
    if not lines or lines[0].strip() != "---":
        return {}
    fm = {}
    for line in lines[1:]:
        if line.strip() == "---":
            break
        if ":" not in line:
            continue
        key, _, raw = line.partition(":")
        val = raw.strip()
        if val.startswith("[") and val.endswith("]"):
            val = [v.strip().strip("'\"") for v in val[1:-1].split(",") if v.strip()]
        elif val.lower() in ("true", "false"):
            val = val.lower() == "true"
        else:
            val = val.strip("'\"")
        fm[key.strip()] = val
    return fm
 
 
def pool_names(root, pool):
    d = root / pool
    return sorted(p.stem for p in d.glob("*.md")) if d.is_dir() else []
 
 
def snapshot(root, inherit_default):
    root = Path(root)
    pools = {p: pool_names(root, p) for p in POOLS}
    agents = {}
    for path in sorted((root / "agents").glob("*.md")):
        fm = read_front_matter(path)
        inherit = fm.get("inheritCustomizations", inherit_default)
        entry = {"inherit": bool(inherit), "declared": "inheritCustomizations" in fm}
        for pool in POOLS:
            explicit = fm.get(pool) or []
            if isinstance(explicit, str):
                explicit = [explicit]
            entry[pool] = (sorted(set(explicit) | set(pools[pool])) if inherit
                           else sorted(set(explicit)))
        agents[fm.get("name", path.stem)] = entry
    return {"inherit_default": inherit_default, "agents": agents}
 
 
def diff(before, after):
    rows = []
    for name in sorted(set(before["agents"]) | set(after["agents"])):
        b, a = before["agents"].get(name), after["agents"].get(name)
        if b is None or a is None:
            rows.append((name, "agent", "added" if b is None else "removed", ""))
            continue
        if b["inherit"] != a["inherit"]:
            rows.append((name, "inherit", str(b["inherit"]), str(a["inherit"])))
        for pool in POOLS:
            for g in sorted(set(a[pool]) - set(b[pool])):
                rows.append((name, pool, "", "+" + g))
            for l in sorted(set(b[pool]) - set(a[pool])):
                rows.append((name, pool, "-" + l, ""))
    return rows
 
 
def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("config_dir", nargs="?")
    ap.add_argument("--inherit-default", choices=["on", "off"], default="on")
    ap.add_argument("--diff", nargs=2, metavar=("BEFORE", "AFTER"))
    args = ap.parse_args()
 
    if args.diff:
        before = json.loads(Path(args.diff[0]).read_text(encoding="utf-8"))
        after = json.loads(Path(args.diff[1]).read_text(encoding="utf-8"))
        rows = diff(before, after)
        if not rows:
            print("no change")
            return 0
        for name, field, was, now in rows:
            print(f"{name}\t{field}\t{was}\t{now}")
        return 1
 
    if not args.config_dir:
        ap.error("config_dir is required")
    print(json.dumps(snapshot(args.config_dir, args.inherit_default == "on"),
                     ensure_ascii=False, indent=2, sort_keys=True))
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

Keys are sorted on output so that ordering jitter never shows up as a fake change in the diff. --diff exits with code 1 when anything moved, which makes it easy to drop into whatever you already run after an update.

You may wonder why the front matter is parsed by hand rather than asked of the agent itself. The check has to work without starting the agent. If you ask the agent what it can see, the answer you get is already shaped by the default you are trying to measure. Rebuilding the picture from the outside, using only the definition files and their neighbours, is the more trustworthy path.

Only One Agent Showed Up in the Diff — the One I Thought Was Narrow

Here is a small reproduction of my layout: four skills, one rule, one subagent, and two agents. The store-copy agent declares inheritCustomizations: false and names the translation glossary. The image-derivative agent declares nothing.

Running the snapshot for the old behaviour and the new one side by side gives this.

$ python3 agent_scope_snapshot.py --inherit-default off /path/to/config > before.json
$ python3 agent_scope_snapshot.py --inherit-default on  /path/to/config > after.json
$ python3 agent_scope_snapshot.py --diff before.json after.json
asset-derive	inherit	False	True
asset-derive	skills		+image-pipeline
asset-derive	skills		+locale-glossary
asset-derive	skills		+review-reply
asset-derive	skills		+seo-audit
asset-derive	rules		+house-style
asset-derive	subagents	+translator

store-copy, which said what it wanted, produces no rows at all. asset-derive, which said nothing, picks up four skills, a rule, and a subagent in one step. Those last two lines are exactly where my unrequested store copy came from.

With a handful of entries you can read this by eye. As the skill pool grows the diff gets long, so I look at the first row for each name first. Whoever has an inherit row is the one who had been living on the default.

Three Places to Look When Reading the Diff

The second row is the one I underestimated. I had assumed extra skills mean nothing worse than extra knowledge sitting unused. But an extra subagent means the agent starts calling other agents on its own. The volume of work changes, which is not remotely comparable to one more skill in the list.

The Order I Use When Making Things Explicit

It is tempting to rewrite every definition at once. I go one row at a time instead, in this order.

1. For every agent with an inherit row, write inheritCustomizations explicitly — even when the value you want matches the new default
2. Then list, by name, the skills and subagents you actually intend to hand over
3. Re-run the snapshot after each edit and confirm that only the intended difference appears
4. Add one line to your own upgrade notes: take a snapshot before and after applying an update

Step three looks like the slow path and turns out to be the fast one. Rewrite everything at once and the reason something got fixed sits in the same diff as the reason something broke. One at a time, they cannot mix.

Deciding whether to turn inheritance on at all is a separate question from noticing that it moved. I wrote up how I make that call — by measuring how much the skill descriptions overlap — in a longer record of that decision, if that is the stage you are stuck at.

One Thing to Do Today

Pick a single agent, the narrowest one you have, and add inheritCustomizations to its definition explicitly. Matching the current default is fine. From the moment it is written down, that agent stops moving when the defaults move.

Since that morning I read blank fields in a definition file differently. They are not "things I have not decided yet" — they are things somebody else is deciding for me. The fewer of them there are, the quieter the mornings after an update.