The Hooks That Get Rejected Are the Cheap Ones: Measuring Matcher Reachability

The empty matcher inverts its own meaning

Look at the first row.

Under substring matching, an empty string hits all 21 tools. Under full matching, it hits none. The same string containing nothing at all swings between "everything" and "nothing" purely on the engine's choice.

People write an empty matcher to mean "don't filter." Under substring semantics that intent survives. Under full-match semantics the entire hook goes quiet. And because the config is not malformed in any way, load-time validation has nothing to complain about.

Writing * turns out to be the more honest mistake. It's a glob habit, and as a regular expression it fails outright with nothing to repeat at position 0. Failing loudly is a courtesy.

Notebook is the sneaky one. It hits NotebookEdit under substring matching and nothing under full matching — exactly the shape that convinces you it works.

Writing Edit was hitting three tools

This was the one in my own config.

search("Edit")    -> Edit, MultiEdit, NotebookEdit
fullmatch("Edit") -> Edit

I had matcher: "Edit" on PostToolUse running Prettier. I meant Edit. In practice it also fired on MultiEdit and NotebookEdit. Two out of every three invocations were unintended.

Notebooks were being handed to Prettier. Nothing broke, thankfully, but that isn't a comfortable place to have been sitting.

As an indie developer working on one machine, nobody was ever going to point this out. The person who wrote the config and the person inconvenienced by it are the same person, which makes the inconvenience invisible.

hookcheck.py — checking reachability statically

Eyeballing doesn't scale, so I wrote a checker. Hand it a config and a list of tool names and it reports unreachable hooks (DEAD) and over-matching ones (WIDE).

It looks at four things: whether the event name is in the known set, whether the matcher compiles, whether it hits at least one tool name, and whether the command's first token resolves to something executable.

That last one I ended up rewriting, for reasons I get to further down. Writing it myself was the single biggest mistake in this whole exercise.

#!/usr/bin/env python3
"""hookcheck.py - statically check hook configurations for reachability"""
import json, os, re, shlex, shutil, sys, time
 
KNOWN_EVENTS = {
    "PreToolUse", "PostToolUse", "UserPromptSubmit",
    "SessionStart", "SessionEnd", "Stop", "Notification",
}
TOOL_EVENTS = {"PreToolUse", "PostToolUse"}
 
 
def load_tool_universe(path):
    with open(path, encoding="utf-8") as fp:
        return [line.strip() for line in fp if line.strip() and not line.startswith("#")]
 
 
def resolve_command(command):
    try:
        argv = shlex.split(command, posix=(os.name != "nt"))
    except ValueError as exc:
        return None, f"shell syntax error: {exc}"
    if not argv:
        return None, "empty command"
    head = argv[0].strip('"')
    found = shutil.which(head)
    if found is None:
        if os.sep in head or (os.altsep and os.altsep in head):
            return None, f"not an executable file: {head}"
        return None, f"not found on PATH: {head}"
    return found, None
 
 
def check(config, tools, semantics="search"):
    findings = []
    for event, groups in config.get("hooks", {}).items():
        if event not in KNOWN_EVENTS:
            findings.append(("DEAD", event, None, f"unknown event name: {event}"))
            continue
        for index, group in enumerate(groups):
            where = f"{event}[{index}]"
            matcher = group.get("matcher")
 
            if event in TOOL_EVENTS:
                if matcher is None:
                    findings.append(("DEAD", where, None, "matcher is undefined"))
                else:
                    try:
                        rx = re.compile(matcher)
                    except re.error as exc:
                        findings.append(("DEAD", where, matcher, f"invalid regex: {exc}"))
                        rx = None
                    if rx is not None:
                        test = rx.fullmatch if semantics == "fullmatch" else rx.search
                        hits = [t for t in tools if test(t)]
                        if not hits:
                            findings.append(("DEAD", where, matcher, "matches no tool name"))
                        elif matcher != ".*" and len(hits) > 1 and not matcher.startswith("^"):
                            findings.append(("WIDE", where, matcher,
                                             f"matches {len(hits)}: {', '.join(hits)}"))
            elif matcher not in (None, "", ".*"):
                findings.append(("DEAD", where, matcher,
                                 f"{event} carries no tool name, so matcher is never evaluated"))
 
            for hook in group.get("hooks", []):
                command = hook.get("command", "")
                _, error = resolve_command(command)
                if error:
                    findings.append(("DEAD", where, command, error))
    return findings
 
 
def hit_counts(config, tools, semantics):
    counts = {}
    for event, groups in config.get("hooks", {}).items():
        if event not in TOOL_EVENTS:
            continue
        for index, group in enumerate(groups):
            matcher = group.get("matcher")
            if matcher is None:
                continue
            try:
                rx = re.compile(matcher)
            except re.error:
                continue
            test = rx.fullmatch if semantics == "fullmatch" else rx.search
            counts[f"{event}[{index}]"] = (matcher, sum(1 for t in tools if test(t)))
    return counts
 
 
def report_divergence(config, tools):
    left = hit_counts(config, tools, "search")
    right = hit_counts(config, tools, "fullmatch")
    diverged = [(w, m, a, right[w][1]) for w, (m, a) in left.items() if right[w][1] != a]
    for where, matcher, a, b in diverged:
        print(f"~ {where} [{matcher}] search={a} fullmatch={b} depends on semantics")
    print(f"\nDIVERGENT={len(diverged)}/{len(left)}")
    return 1 if diverged else 0
 
 
def main():
    if len(sys.argv) < 3:
        print("usage: hookcheck.py <hooks.json> <tools.txt> [--fullmatch|--both]", file=sys.stderr)
        return 2
    with open(sys.argv[1], encoding="utf-8") as fp:
        config = json.load(fp)
    tools = load_tool_universe(sys.argv[2])
 
    if "--both" in sys.argv:
        return report_divergence(config, tools)
 
    semantics = "fullmatch" if "--fullmatch" in sys.argv else "search"
    started = time.perf_counter()
    findings = check(config, tools, semantics)
    elapsed_ms = (time.perf_counter() - started) * 1000
 
    dead = sum(1 for f in findings if f[0] == "DEAD")
    for level, where, subject, message in findings:
        mark = "x" if level == "DEAD" else "!"
        subject_text = f" [{subject}]" if subject else ""
        print(f"{mark} {where}{subject_text} {message}")
    print(f"\nDEAD={dead} WIDE={len(findings) - dead} semantics={semantics} {elapsed_ms:.2f}ms")
    return 1 if dead else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

The tool list is just one name per line. If writing it out by hand feels tedious, copy the tool listing from the start of a session and save it.

The not matcher.startswith("^") condition on the over-match warning exists because a matcher anchored at the front reads as a deliberate attempt to cast a wide net. Warning about intentional breadth is the fastest way to teach people to ignore warnings.

Running it against my own config

Here is what my config produced.

$ python3 hookcheck.py hooks.json tools.txt
! PreToolUse[0] [Bash] matches 2: Bash, BashOutput
x PreToolUse[1] [*] invalid regex: nothing to repeat at position 0
x PreToolUse[2] [edit] matches no tool name
x PreToolUse[2] [prettier --write] not found on PATH: prettier
! PostToolUse[0] [Edit] matches 3: Edit, MultiEdit, NotebookEdit
! PostToolUse[1] matches 21: Bash, BashOutput, KillShell, Read, Write, ...
x PostToolUse[1] [notify-send done] not found on PATH: notify-send
x UserPromptSubmit[0] [Bash] UserPromptSubmit carries no tool name, so matcher is never evaluated
x PreCompact unknown event name: PreCompact

DEAD=6 WIDE=3 semantics=search 0.62ms

And with --fullmatch, changing only the semantics:

$ python3 hookcheck.py hooks.json tools.txt --fullmatch
...
DEAD=7 WIDE=0 semantics=fullmatch 0.51ms

DEAD went from 6 to 7, WIDE from 3 to 0. Not a single character of the config changed.

I keep both invocations around permanently and read only the diff between them. Any matcher whose verdict moves is a matcher leaning on an assumption about the engine. Since the semantics aren't spelled out in the documentation, leaning on them is a bet I'd rather not place.

Subtracting DEAD counts by eye every morning turned out to be a sloppy way to read that, so I added --both, which prints the divergence directly.

$ python3 hookcheck.py hooks.json tools.txt --both
~ PreToolUse[0] [Bash] search=2 fullmatch=1 depends on semantics
~ PostToolUse[0] [Edit] search=3 fullmatch=1 depends on semantics
~ PostToolUse[1] [] search=21 fullmatch=0 depends on semantics

DIVERGENT=3/4

Three of the four groups that compiled changed their hit count on semantics alone. That's the number worth tracking — not the DEAD total, but how many matchers change meaning depending on the engine's mood.

The rejected ones are the cheap ones

This was the part that surprised me.

A dead hook costs 0ms. It never runs, so it consumes nothing. Its only damage is structural — a quality check you believed was running isn't. Load-time rejection makes that visible immediately, which is exactly the right treatment.

Over-matching is a different animal. It runs. It runs every time. A hook is a process spawn, so real wall-clock time attaches to it.

I timed four no-op commands, 30 launches each, same machine.

A Node process that does nothing costs 31.61ms. A real hook adds its own work on top of that.

With matcher: "Edit" on a Node-based formatter, two of every three firings were unintended. An empty matcher under substring semantics rides along on every tool call: 1.6 seconds across a 50-call session, 6.3 seconds across 200. Simple arithmetic, but it lands in your day as "things feel sluggish today" rather than as a number you can point at.

I suspected the matching itself might be the cost. It isn't.

Even a thousand matchers costs 0.19ms. Every bit of the expense sits on the process-spawn side.

Which means load-time validation catches the cheap failure. The expensive one walks straight past it. A config being valid and a config being what you meant turn out to be entirely separate claims.

Cheap enough for a CI step

A checker only survives if it's fast. I varied the group count.

At 1,000 groups it's 48.28ms of checking and 76.1ms including Python startup. At any size a human will actually write by hand, treat it as free.

It exits 1 when there is at least one DEAD finding, so it stops a pipeline on its own.

python3 tools/hookcheck.py .antigravity/hooks.json tools.txt \
  && python3 tools/hookcheck.py .antigravity/hooks.json tools.txt --both

I run the reachability check and the divergence check back to back. A config that only passes under one set of semantics will quietly change behavior the day the environment does.

WIDE findings deliberately don't affect the exit code. How much over-matching a repository tolerates is a local decision. Having them in front of me on every run has been enough.

The pass I make once per repository

Every time I pick up a new repository, I go through this. It takes under five minutes.

1. Dump the tool listing from the start of a session into tools.txt, one name per line. Do it after the MCP servers have connected, or their names won't be there
2. Run hookcheck.py under substring semantics and drive DEAD to zero. Most of what surfaces here is spelling and event names
3. Run it again with --both and confirm DIVERGENT=0. Wrap any matcher it names in ^ and $
4. Read the tool names on each WIDE line one by one and check that nothing unintended snuck in. This is the step to slow down on
5. Rewrite hook commands to use absolute paths or an explicit npx / env, then repeat steps 2 and 3
6. Wire both invocations into CI so the next person to touch the config doesn't have to remember any of this

Step 4 is the one a machine can't do for you. Only the author knows what was intended.

"It's on PATH" was too generous a test

This part is about rewriting the checker itself.

My first resolve_command walked the PATH directories in order and returned the first candidate where os.access(candidate, os.X_OK) came back true. It felt like the straightforward implementation.

It was too straightforward. Files aren't the only things carrying an execute bit. Directories carry one too — on a directory it means "you may traverse this" — so an ordinary directory answers true.

I reproduced it locally by dropping a directory named prettier at the front of PATH. Nothing else.

The old implementation was reporting directories as runnable commands. Worse, that error pushes DEAD counts down, not up. The checker says all clear and the hook fails to launch — the exact direction of wrongness a checker must never have.

Adding a parent directory to PATH when you meant node_modules/.bin is a common enough slip. I saw DEAD=0, relaxed, and found a week later that the hook had never run. The config was fine. The thing checking it was not.

A false negative in a checker is worse than having no checker. When you know nothing is watching, you look yourself. When you think something is watching, nobody does.

The rewrite hands the job to shutil.which, which also tests that the candidate is a file, so the directory above gets rejected. On Windows it consults PATHEXT to fill in extensions, so prettier can resolve to prettier.cmd. My hand-rolled PATH walk dropped that branch entirely.

shlex.split needed the same treatment. Its default POSIX mode consumes backslashes as escape characters.

shlex.split(r'C:\tools\fmt.exe --write')
  -> ['C:toolsfmt.exe', '--write']
shlex.split(r'C:\tools\fmt.exe --write', posix=False)
  -> ['C:\\tools\\fmt.exe', '--write']

A path with its separators eaten resolves nowhere, so every Windows hook lines up as DEAD. That direction is at least visible — but real DEAD findings get buried in the noise, which is its own kind of damage.

I measured what the rewrite costs. Per command, the os.access walk runs at a median of 0.032ms and shutil.which at 0.046ms. The difference is 0.014ms. Even across 1,000 groups that's 14ms on top, which disappears into the check times measured above. If correctness costs 14ms, there is no argument against buying it.

What the documentation doesn't say

Three things I wish I'd known before editing any of this.

The matching semantics are frequently unspecified. Substring versus full match changes the meaning of an empty matcher, of Edit, and of Notebook. When in doubt, check under both and avoid any matcher where the verdict moves. Wrapping the pattern in ^ and $ makes both engines agree.

Command resolution depends on PATH — and on how you test it. prettier resolved on my machine and not in CI. The reverse happens too. Use absolute paths in hook commands, or put npx or env in front explicitly, and let shutil.which do the lookup rather than walking PATH by hand. The checker reads the live PATH, which is precisely why running it inside CI is worth the step.

Unknown event names can fail quietly. A plausible-looking PreCompact does nothing if it isn't in the known set. Comparing against a known-event list is the least interesting check in the script, and it's the one that actually caught me.

For how registration timing relates to firing conditions, I wrote up the move to declarative session definitions in the hooks that went from flaky to failing every time. Reading both makes it easier to see where each decision gets made.

I changed what I suspect first

I used to suspect the config whenever a hook didn't fire. Now I start with the hooks that do fire.

The range of problems caught at load time has genuinely widened. Still, not being rejected is not evidence of being correct. Counting how many tools a matcher actually hits was the one step that closed that gap.

Run your own hooks.json through both semantics once. Even with DEAD at zero, if the WIDE lines list tool names you don't recognize, that's where you're paying 31ms a call.