What @ Includes Actually Shrink When You Split Agent Rule Files

Returning an empty string for a missing include is the dangerous choice

There is a fix, and it lives in your expander rather than in your file layout: never return an empty string when a target is missing.

My expander leaves a marker instead.

if not os.path.isfile(real):
    stat["missing"].append(path)
    return f"<!-- MISSING INCLUDE: {path} -->\n", stat

Three lines, but they change how the failure is discovered. The marker catches a human skimming the expanded text, and the missing array catches an automated check. Either path works.

The same reasoning applies to a depth ceiling, as insurance for when you have picked a base that cannot detect cycles.

if depth > max_depth:
    raise Cycle(f"exceeded {max_depth} levels: {os.path.relpath(real, root)}")

Cycle detection and depth limiting look redundant. They cover each other's blind spots, and both are cheap.

The full expander

This is what I run before handing rule files to the agent, and again whenever I edit them.

#!/usr/bin/env python3
"""Expand @path/to/file includes and audit the result before the agent sees it.
 
base="file"  ... relative to the including file (the base moves as you nest)
base="root"  ... fixed to the project root (same string, same meaning, anywhere)
"""
import sys, re, os, json
 
# Only a full line consisting of @path counts as an include. This keeps
# @mentions and email addresses in prose from being swallowed.
INCLUDE = re.compile(r'^([ \t]*)@([A-Za-z0-9_./\-]+\.(?:md|txt|json))[ \t]*$')
 
 
class Cycle(Exception):
    pass
 
 
def expand(path, root, base="root", stack=None, stat=None, depth=0, max_depth=16):
    if stack is None:
        stack = []
        stat = {"count": {}, "missing": []}
 
    real = os.path.realpath(os.path.join(root, path))
 
    # Normalize with realpath before comparing, so a symlink or a ../ detour
    # to the same file is not treated as a different file.
    if real in stack:
        chain = [os.path.relpath(p, root) for p in stack[stack.index(real):]]
        raise Cycle(" -> ".join(chain + [os.path.relpath(real, root)]))
    if depth > max_depth:
        raise Cycle(f"exceeded {max_depth} levels: {os.path.relpath(real, root)}")
 
    stat["count"][real] = stat["count"].get(real, 0) + 1
 
    # Never return empty for a missing target. Rules disappearing in silence
    # is the failure you will not notice.
    if not os.path.isfile(real):
        stat["missing"].append(path)
        return f"<!-- MISSING INCLUDE: {path} -->\n", stat
 
    out = []
    for line in open(real, encoding="utf-8"):
        m = INCLUDE.match(line)
        if not m:
            out.append(line)
            continue
        indent, target = m.group(1), m.group(2)
        anchor = "" if base == "root" else os.path.dirname(os.path.relpath(real, root))
        child, stat = expand(os.path.join(anchor, target), root, base,
                             stack + [real], stat, depth + 1, max_depth)
        # Propagate the including line's indentation into the child, so an
        # include placed inside a bulleted list does not flatten the nesting.
        out.extend(indent + l if l.strip() else l for l in child.splitlines(keepends=True))
 
    return "".join(out), stat
 
 
def audit(entry, root, base="root"):
    try:
        text, stat = expand(entry, root, base)
    except Cycle as e:
        return {"entry": entry, "ok": False, "error": f"circular include: {e}"}
    return {
        "entry": entry, "ok": True, "base": base,
        "chars": len(text),
        "files": len(stat["count"]),
        "missing": stat["missing"],
        "duplicated": {os.path.relpath(p, root): n
                       for p, n in stat["count"].items() if n > 1},
    }
 
 
if __name__ == "__main__":
    root, base = sys.argv[1], sys.argv[2]
    for entry in sys.argv[3:]:
        print(json.dumps(audit(entry, root, base), ensure_ascii=False))

Run it like this:

python3 expand.py . root AGENTS.md

Anchoring the pattern to a full line is deliberate. @ shows up constantly in prose, and a partial match will treat mid-sentence text as an include. My first version matched at line start only, and it swallowed explanatory text inside a bulleted list. Full line, trailing whitespace allowed, nothing else.

Pulling a shared file in through two paths doubles it, exactly as you would expect

There is a second trap, and this one makes things bigger.

As you split further, you end up with intermediate files that include the shared file. entry.md includes x.md and y.md, and both include common.md. Your includes are now a graph, not a tree.

I measured it with a 3,200-byte common.md:

{"entry": "entry.md", "ok": true, "base": "root", "chars": 2416,
 "files": 4, "missing": [], "duplicated": {"rules/common.md": 2}}

duplicated reports 2. The expanded output contains common.md twice, verbatim. The agent reads the same rules twice because you handed it the same rules twice.

The deeper your split, the easier this shape is to create. You can end up with more expanded input after a reorganization intended to reduce it. That is why the expander reports duplicated at all.

My fix was to pull shared files in at exactly one place, the entry point, and never from an intermediate file. Intermediate files are written assuming the shared rules are already present. Since a file cannot verify that assumption on its own, I leave a one-line comment at the top of each dependent file naming where it expects to be included from.

Where to split and where to leave things alone

Doing this for real surfaced a few usable boundaries.

Worth splitting

• Rules that are word-for-word identical across repositories. In my measurement that was the 695-line block.
• Supplementary material whose relevance depends on the task. Sessions that do not need it stop paying for it.
• Content with a different update owner. If only one side changes often, the diffs get much easier to read.

Better left in one file

• Rules where ordering carries meaning. Shift the include position and you shift how precedence reads.
• Conditional rules that get misread without their surrounding context. An "except when..." clause stranded in another file is a hazard.
• Blocks under a hundred lines. The file-management overhead exceeds the benefit.

I overshot on that last one and rolled it back. Fine-grained splitting feels tidy, but an entry file that is nothing but a column of @ lines tells you nothing when you open it. Keeping files readable on their own turned out to be faster overall. When you are the only person reviewing your own repositories, that difference lands directly on your working hours.

For the decision order: first check whether the same text exists in more than one place. If not, do not split. If it does, check whether that block stands on its own when read in isolation. If it does not, widen the boundary until it does, then extract.

Combining this with subdirectory config files

2.11.0 also added discovery of skills.json, agents.json, and rules.json from subdirectories, so a project with multiple packages can give each package its own rules.

That solves a different problem than @ does. Includes govern how a single rule file is composed. Subdirectory config governs which rule file gets selected in the first place.

Here is how I divide them:

That third one lets a custom Markdown agent point directly at a rule file, which is useful when different agents follow different conventions and you want that mapping written down.

The thing to watch when combining them is the same rule arriving through two routes. If a rule file loaded via a subdirectory rules.json is also pulled in by an @, you are back in the double-expansion case above. The expander only counts the include path, so this one has to be prevented at design time.

Measure before you restructure

If you are considering a split, measure first. Counting duplicate lines across your files takes one command:

cat rules-a.md rules-b.md rules-c.md | grep -v '^\s*$' \
  | sort | uniq -c | sort -rn | awk '$1>1' | wc -l

A small number means splitting buys you nothing. A large number means you can proceed knowing what you are actually reducing. In my case 2,216 of 3,430 lines were duplicates, and knowing that number first produced a very different design than the one I had in mind.

Thanks for reading. I hope this saves you a debugging session.