I Copied the Same agent.md Into Another Repo and It Quietly Did a Different Job

The Naive Checker Turned 7 of 8 Repos Red

My first attempt was thirty lines written straight through. Read mcp_config.json, then confirm each entry in requiredMcpTools appears there.

#!/usr/bin/env python3
"""naive.py — the naive resolver. It only looks at serverUrl."""
import json, os, glob, yaml
 
def load_agent(path):
    raw = open(path).read()
    _, fm, body = raw.split("---", 2)
    return yaml.safe_load(fm), body
 
def servers_of(cfg_path):
    cfg = json.load(open(cfg_path))
    out = {}
    for name, spec in cfg.get("mcpServers", {}).items():
        if "serverUrl" not in spec:      # <- the naive part
            continue
        out[name] = set(spec.get("tools", []))
    return out
 
def check(repo):
    fm, _ = load_agent(os.path.join(repo, ".antigravity/agents/release-notes.md"))
    srv = servers_of(os.path.join(repo, ".antigravity/mcp_config.json"))
    missing = []
    for ref in fm.get("requiredMcpTools", []):
        s, t = ref.split(".", 1)
        if s not in srv or t not in srv[s]:
            missing.append(ref)
    return missing
 
if __name__ == "__main__":
    for repo in sorted(glob.glob("/tmp/exp/fixture/*")):
        m = check(repo)
        print(f"{os.path.basename(repo):20s} {'OK' if not m else 'MISSING ' + ','.join(m)}")

The output:

calm-sounds          MISSING slack.post_message
calm-sounds-web      MISSING github.list_releases,github.create_release,slack.post_message
photo-frame          MISSING github.list_releases,github.create_release
photo-frame-tv       MISSING github.list_releases,github.create_release,slack.post_message
sleep-timer          MISSING github.list_releases,github.create_release
sleep-timer-mac      MISSING github.list_releases,github.create_release,slack.post_message
wallpaper-android    MISSING github.list_releases,github.create_release,slack.post_message
wallpaper-ios        OK

Seven of eight red. Seventeen findings in total.

For a moment my stomach dropped. Had my assumptions really rotted this badly across repos?

Working through them one by one told a different story. wallpaper-android has both github and slack. The tool names match exactly. It came back fully red because its mcp_config.json spells the URL key url rather than serverUrl.

One hasty line of mine had condemned an entire repository.

A Compatibility Alias Became Noise on the Tooling Side

As part of the MCP stabilization work, mcp_config.json now accepts url in addition to serverUrl. Adding the plainer spelling without breaking existing configs is a reasonable call, and for anyone writing configuration it is a small kindness.

The trouble starts when you write your own tooling that reads that configuration.

A reader that only recognizes one spelling is syntactically correct and semantically wrong. Worse, it fails as "server does not exist," which points you nowhere near the real cause. The server is there. You just are not looking at it.

Two more variants of the same shape showed up.

None of these are environment problems. They are spelling problems.

Spelling problems can be normalized away, and they have to be, because otherwise the real gaps get buried. Not knowing how many of seventeen findings are genuine leaves you with a checker that is no better than none.

Normalize First, Then Resolve

The rewrite is preflight.py. It does three things: normalize the configuration, resolve the declared capabilities against the target repo, and exit non-zero if anything fails to resolve.

#!/usr/bin/env python3
"""preflight.py — resolve an agent.md capability contract against the target repo."""
from __future__ import annotations
import json, os, sys, glob, time, yaml
from dataclasses import dataclass, field
 
URL_KEYS = ("url", "serverUrl")          # both are valid as of 1.1.6
SERVER_ALIASES = {"gh": "github", "github": "github", "slack": "slack"}
 
@dataclass
class Finding:
    kind: str        # mcp-server / mcp-tool / command
    ref: str
    detail: str
 
@dataclass
class Report:
    repo: str
    findings: list[Finding] = field(default_factory=list)
    @property
    def blocked(self) -> bool:
        return bool(self.findings)
 
def read_agent(path: str) -> dict:
    raw = open(path, encoding="utf-8").read()
    if not raw.startswith("---"):
        raise ValueError(f"{path}: no frontmatter")
    _, fm, _body = raw.split("---", 2)
    return yaml.safe_load(fm) or {}
 
def normalize_servers(cfg_path: str) -> dict[str, set[str]]:
    """Collapse serverUrl and url into a single shape."""
    cfg = json.load(open(cfg_path, encoding="utf-8"))
    out: dict[str, set[str]] = {}
    for raw_name, spec in cfg.get("mcpServers", {}).items():
        if not any(k in spec for k in URL_KEYS):
            continue                      # stdio-launched servers are out of scope
        name = SERVER_ALIASES.get(raw_name, raw_name)
        out.setdefault(name, set()).update(spec.get("tools", []))
    return out
 
def snake(s: str) -> str:
    """Treat listReleases and list_releases as the same tool."""
    return "".join("_" + c.lower() if c.isupper() else c for c in s).lstrip("_")
 
def check(repo: str) -> Report:
    rep = Report(repo=os.path.basename(repo))
    fm = read_agent(os.path.join(repo, ".antigravity/agents/release-notes.md"))
    servers = normalize_servers(os.path.join(repo, ".antigravity/mcp_config.json"))
 
    for ref in fm.get("requiredMcpTools", []):
        raw_srv, tool = ref.split(".", 1)
        srv = SERVER_ALIASES.get(raw_srv, raw_srv)
        if srv not in servers:
            rep.findings.append(Finding("mcp-server", ref, f"server {srv} is not defined"))
            continue
        have = {snake(t) for t in servers[srv]}
        if snake(tool) not in have:
            rep.findings.append(Finding("mcp-tool", ref, f"{srv} has no {tool}"))
 
    if fm.get("commandExecutionPolicy") == "allowlist":
        available = set(json.load(open(os.path.join(repo, "commands.json"))))
        for cmd in fm.get("allowedCommands", []):
            head = cmd.split()[0]
            if head not in available:
                rep.findings.append(Finding("command", cmd, f"{head} is absent here"))
    return rep
 
def main(paths: list[str]) -> int:
    t0 = time.perf_counter()
    reports = [check(p) for p in sorted(paths)]
    elapsed = (time.perf_counter() - t0) * 1000
    for r in reports:
        if not r.blocked:
            print(f"OK    {r.repo:20s} resolved")
        else:
            print(f"BLOCK {r.repo:20s} {len(r.findings)} finding(s)")
            for f in r.findings:
                print(f"        [{f.kind:11s}] {f.ref} — {f.detail}")
    blocked = sum(1 for r in reports if r.blocked)
    kinds: dict[str, int] = {}
    for r in reports:
        for f in r.findings:
            kinds[f.kind] = kinds.get(f.kind, 0) + 1
    print(f"\n{len(reports)} repos / blocked {blocked} / {elapsed:.1f} ms")
    print("by kind:", ", ".join(f"{k}={v}" for k, v in sorted(kinds.items())))
    return 1 if blocked else 0
 
if __name__ == "__main__":
    sys.exit(main(sys.argv[1:] or glob.glob("/tmp/exp/fixture/*")))

Three design choices carried the weight here.

1. Keep normalization and judgment apart. normalize_servers only reshapes configuration; it never decides whether something is acceptable. Judgment stays inside check. Mixing them means touching the judgment path every time you add an alias.
2. Give every gap a kind. A missing server, a missing tool, and a missing command call for completely different fixes, so Finding.kind separates them.
3. Speak through the exit code. A single unresolved item returns 1. On unattended runs, that one line is the only thing standing between you and a silent partial success.

snake is a tiny function, but it is the piece I would most recommend keeping. Tool naming belongs to whoever implemented the server. Absorbing that on your side ages far better than rewriting your declarations every time a server changes its conventions.

The Numbers

Across the eight repos:

BLOCK calm-sounds          1 finding(s)
        [mcp-server ] slack.post_message — server slack is not defined
BLOCK calm-sounds-web      1 finding(s)
        [mcp-tool   ] github.create_release — github has no create_release
OK    photo-frame          resolved
BLOCK photo-frame-tv       1 finding(s)
        [command    ] npm run build — npm is absent here
OK    sleep-timer          resolved
OK    sleep-timer-mac      resolved
OK    wallpaper-android    resolved
OK    wallpaper-ios        resolved
 
8 repos / blocked 3 / 10.9 ms
by kind: command=1, mcp-server=1, mcp-tool=1

Side by side, with runtimes reported as the median of 20 runs:

Of the seventeen findings, three pointed at genuine environment gaps. The remaining fourteen — 82.4% — were spelling drift.

Normalization cost 0.83 ms across all eight repos combined.

Scale did not change the picture. Replicating the same structure 30 times gave 240 repos; the median of 5 runs came to 318 ms, with 90 repos blocked. The whole tree occupies 5.7 MB on disk.

That is short enough to run on every commit without noticing it. Checks of this kind get removed when they are slow. At 0.3 seconds there is no reason to remove it.

The Real Blocker Was Not in MCP

This was the part that genuinely surprised me.

photo-frame-tv was blocked because npm is missing. Both MCP servers are present. Every tool name matches. What was absent was the thing behind npm run build in allowedCommands.

And the naive checker never finds it, because it only ever reads requiredMcpTools.

So the noisier tool produced seventeen findings while missing the one gap that would actually have stalled an unattended run. The blind spot was in the loud checker, not the quiet one.

My expectation going in had been the opposite. I assumed MCP would be the fragile surface and that commands would mostly be present anywhere. In reality I do keep repos without Node installed, and that bit first.

The lesson I took away is that finding count and coverage are unrelated quantities. A long list feels reassuring, but the length may live entirely in the noise. Measuring coverage means counting what you look at, broken down by kind — which is exactly why the by-kind summary is part of the output. That line is a health check on the checker itself.

There is a second lesson. An alias added for compatibility becomes noise for anyone writing tooling on top. A change that is kind to config authors hands a small debt to config readers. Where the specification grows permissive, keep the normalization on your side. That division has held up well for me.

Places to Watch

Both the traps I fell into and the ones I nearly did.

Reading inheritMcp: true as a guarantee

What gets inherited is configuration, not capability. The awkward part is how easily that true becomes a reason to skip verification. I did precisely this on my first port.

Scattering the alias table into the judgment path

I was tempted to inline SERVER_ALIASES inside check. Doing so means every new alias adds a branch to the judgment logic. Kept in normalization, adding one is a single dictionary line.

Applying snake everywhere

Tool-name normalization is used only for comparison, never for display. Printing a normalized name shows the reader a spelling that appears nowhere in their config, which makes tracing the cause harder. Error messages keep the original.

Silently dropping stdio servers

normalize_servers skips definitions without a URL. If you run servers as local processes, that becomes a quiet blind spot. Nothing in my environment matched, so I accepted the gap for now; if yours does, add an existence check for the launch command.

Letting the check grow heavy enough to be removed

This one is operational rather than technical. A pre-adoption check that runs slowly will be dropped eventually. The 318 ms figure across 240 repos is, in that sense, a budget as much as a measurement.

Where I Stop

This preflight only asks whether declared assumptions resolve in the target repo. It says nothing about whether the agent's output is any good.

Going further means entering territory you cannot see without running the thing, and trying to verify runtime behavior before runtime rarely ends well. I would rather verify a small set of things completely.

My current habit is simple: after copying an agent.md into another repository, run this one command. If it does not pass, I either fix mcp_config.json or bring allowedCommands in line with what that repo actually has. The agent runs afterward.

Late-night copies showing up as morning surprises have become noticeably rarer.

Next on my list is replacing commands.json with real shutil.which probing, and surfacing blocked findings as PR comments. The first is straightforward. The second needs a decision about how much the tooling should fix on its own, so it can wait.

If you carry definition files between projects, adding a declaration like requiredMcpTools is worth trying on its own. Inventing a key the specification does not define felt presumptuous at first. Having more of the assumptions readable before the run turned out to matter more than I expected.

Thanks for reading — I am still early in this, and the alias table in particular is something I am figuring out as I go.