The Hooks That Get Rejected Are the Cheap Ones: Measuring Matcher Reachability

The empty matcher inverts its own meaning

Look at the first row.

Under substring matching, an empty string hits all 21 tools. Under full matching, it hits none. The same string containing nothing at all swings between "everything" and "nothing" purely on the engine's choice.

People write an empty matcher to mean "don't filter." Under substring semantics that intent survives. Under full-match semantics the entire hook goes quiet. And because the config is not malformed in any way, load-time validation has nothing to complain about.

Writing * turns out to be the more honest mistake. It's a glob habit, and as a regular expression it fails outright with nothing to repeat at position 0. Failing loudly is a courtesy.

Notebook is the sneaky one. It hits NotebookEdit under substring matching and nothing under full matching — exactly the shape that convinces you it works.

Writing Edit was hitting three tools

This was the one in my own config.

search("Edit")    -> Edit, MultiEdit, NotebookEdit
fullmatch("Edit") -> Edit

I had matcher: "Edit" on PostToolUse running Prettier. I meant Edit. In practice it also fired on MultiEdit and NotebookEdit. Two out of every three invocations were unintended.

Notebooks were being handed to Prettier. Nothing broke, thankfully, but that isn't a comfortable place to have been sitting.

As an indie developer working on one machine, nobody was ever going to point this out. The person who wrote the config and the person inconvenienced by it are the same person, which makes the inconvenience invisible.

hookcheck.py — checking reachability statically

Eyeballing doesn't scale, so I wrote a checker. Hand it a config and a list of tool names and it reports unreachable hooks (DEAD) and over-matching ones (WIDE).

It looks at four things: whether the event name is in the known set, whether the matcher compiles, whether it hits at least one tool name, and whether the command's first token resolves to something executable.

#!/usr/bin/env python3
"""hookcheck.py - statically check hook configurations for reachability"""
import json, os, re, shlex, sys, time
 
KNOWN_EVENTS = {
    "PreToolUse", "PostToolUse", "UserPromptSubmit",
    "SessionStart", "SessionEnd", "Stop", "Notification",
}
TOOL_EVENTS = {"PreToolUse", "PostToolUse"}
 
 
def load_tool_universe(path):
    with open(path, encoding="utf-8") as fp:
        return [line.strip() for line in fp if line.strip() and not line.startswith("#")]
 
 
def resolve_command(command):
    try:
        argv = shlex.split(command)
    except ValueError as exc:
        return None, f"shell syntax error: {exc}"
    if not argv:
        return None, "empty command"
    head = argv[0]
    if "/" in head:
        return (head, None) if os.access(head, os.X_OK) else (None, f"not executable: {head}")
    for directory in os.environ.get("PATH", "").split(os.pathsep):
        candidate = os.path.join(directory, head)
        if os.access(candidate, os.X_OK):
            return candidate, None
    return None, f"not found on PATH: {head}"
 
 
def check(config, tools, semantics="search"):
    findings = []
    for event, groups in config.get("hooks", {}).items():
        if event not in KNOWN_EVENTS:
            findings.append(("DEAD", event, None, f"unknown event name: {event}"))
            continue
        for index, group in enumerate(groups):
            where = f"{event}[{index}]"
            matcher = group.get("matcher")
 
            if event in TOOL_EVENTS:
                if matcher is None:
                    findings.append(("DEAD", where, None, "matcher is undefined"))
                else:
                    try:
                        rx = re.compile(matcher)
                    except re.error as exc:
                        findings.append(("DEAD", where, matcher, f"invalid regex: {exc}"))
                        rx = None
                    if rx is not None:
                        test = rx.fullmatch if semantics == "fullmatch" else rx.search
                        hits = [t for t in tools if test(t)]
                        if not hits:
                            findings.append(("DEAD", where, matcher, "matches no tool name"))
                        elif matcher != ".*" and len(hits) > 1 and not matcher.startswith("^"):
                            findings.append(("WIDE", where, matcher,
                                             f"matches {len(hits)}: {', '.join(hits)}"))
            elif matcher not in (None, "", ".*"):
                findings.append(("DEAD", where, matcher,
                                 f"{event} carries no tool name, so matcher is never evaluated"))
 
            for hook in group.get("hooks", []):
                command = hook.get("command", "")
                _, error = resolve_command(command)
                if error:
                    findings.append(("DEAD", where, command, error))
    return findings
 
 
def main():
    if len(sys.argv) < 3:
        print("usage: hookcheck.py <hooks.json> <tools.txt> [--fullmatch]", file=sys.stderr)
        return 2
    semantics = "fullmatch" if "--fullmatch" in sys.argv else "search"
    with open(sys.argv[1], encoding="utf-8") as fp:
        config = json.load(fp)
    tools = load_tool_universe(sys.argv[2])
 
    started = time.perf_counter()
    findings = check(config, tools, semantics)
    elapsed_ms = (time.perf_counter() - started) * 1000
 
    dead = sum(1 for f in findings if f[0] == "DEAD")
    for level, where, subject, message in findings:
        mark = "x" if level == "DEAD" else "!"
        subject_text = f" [{subject}]" if subject else ""
        print(f"{mark} {where}{subject_text} {message}")
    print(f"\nDEAD={dead} WIDE={len(findings) - dead} semantics={semantics} {elapsed_ms:.2f}ms")
    return 1 if dead else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

The tool list is just one name per line. If writing it out by hand feels tedious, copy the tool listing from the start of a session and save it.

The not matcher.startswith("^") condition on the over-match warning exists because a matcher anchored at the front reads as a deliberate attempt to cast a wide net. Warning about intentional breadth is the fastest way to teach people to ignore warnings.

Running it against my own config

Here is what my config produced.

$ python3 hookcheck.py hooks.json tools.txt
! PreToolUse[0] [Bash] matches 2: Bash, BashOutput
x PreToolUse[1] [*] invalid regex: nothing to repeat at position 0
x PreToolUse[2] [edit] matches no tool name
x PreToolUse[2] [prettier --write] not found on PATH: prettier
! PostToolUse[0] [Edit] matches 3: Edit, MultiEdit, NotebookEdit
! PostToolUse[1] matches 21: Bash, BashOutput, KillShell, Read, Write, ...
x PostToolUse[1] [notify-send done] not found on PATH: notify-send
x UserPromptSubmit[0] [Bash] UserPromptSubmit carries no tool name, so matcher is never evaluated
x PreCompact unknown event name: PreCompact

DEAD=6 WIDE=3 semantics=search 0.62ms

And with --fullmatch, changing only the semantics:

$ python3 hookcheck.py hooks.json tools.txt --fullmatch
...
DEAD=7 WIDE=0 semantics=fullmatch 0.51ms

DEAD went from 6 to 7, WIDE from 3 to 0. Not a single character of the config changed.

I keep both invocations around permanently and read only the diff between them. Any matcher whose verdict moves is a matcher leaning on an assumption about the engine. Since the semantics aren't spelled out in the documentation, leaning on them is a bet I'd rather not place.

The rejected ones are the cheap ones

This was the part that surprised me.

A dead hook costs 0ms. It never runs, so it consumes nothing. Its only damage is structural — a quality check you believed was running isn't. Load-time rejection makes that visible immediately, which is exactly the right treatment.

Over-matching is a different animal. It runs. It runs every time. A hook is a process spawn, so real wall-clock time attaches to it.

I timed four no-op commands, 30 launches each, same machine.

A Node process that does nothing costs 31.61ms. A real hook adds its own work on top of that.

With matcher: "Edit" on a Node-based formatter, two of every three firings were unintended. An empty matcher under substring semantics rides along on every tool call: 1.6 seconds across a 50-call session, 6.3 seconds across 200. Simple arithmetic, but it lands in your day as "things feel sluggish today" rather than as a number you can point at.

I suspected the matching itself might be the cost. It isn't.

Even a thousand matchers costs 0.19ms. Every bit of the expense sits on the process-spawn side.

Which means load-time validation catches the cheap failure. The expensive one walks straight past it. A config being valid and a config being what you meant turn out to be entirely separate claims.

Cheap enough for a CI step

A checker only survives if it's fast. I varied the group count.

At 1,000 groups it's 48.28ms of checking and 76.1ms including Python startup. At any size a human will actually write by hand, treat it as free.

It exits 1 when there is at least one DEAD finding, so it stops a pipeline on its own.

python3 tools/hookcheck.py .antigravity/hooks.json tools.txt \
  && python3 tools/hookcheck.py .antigravity/hooks.json tools.txt --fullmatch

I invoke it twice, once per semantics. A config that only passes under one of them will quietly change behavior the day the environment does.

WIDE findings deliberately don't affect the exit code. How much over-matching a repository tolerates is a local decision. Having them in front of me on every run has been enough.

The pass I make once per repository

Every time I pick up a new repository, I go through this. It takes under five minutes.

1. Dump the tool listing from the start of a session into tools.txt, one name per line. Do it after the MCP servers have connected, or their names won't be there
2. Run hookcheck.py under substring semantics and drive DEAD to zero. Most of what surfaces here is spelling and event names
3. Run the same config with --fullmatch and confirm the DEAD count is unchanged. If it moves, wrap that matcher in ^ and $
4. Read the tool names on each WIDE line one by one and check that nothing unintended snuck in
5. Rewrite hook commands to use absolute paths or an explicit npx / env, then repeat steps 2 and 3
6. Wire both invocations into CI so the next person to touch the config doesn't have to remember any of this

Step 4 is the one a machine can't do for you. Only the author knows what was intended.

What the documentation doesn't say

Three things I wish I'd known before editing any of this.

The matching semantics are frequently unspecified. Substring versus full match changes the meaning of an empty matcher, of Edit, and of Notebook. When in doubt, check under both and avoid any matcher where the verdict moves. Wrapping the pattern in ^ and $ makes both engines agree.

Command resolution depends on PATH. prettier resolved on my machine and not in CI. The reverse happens too. Use absolute paths in hook commands, or put npx or env in front explicitly. The checker reads the live PATH, which is precisely why running it inside CI is worth the step.

Unknown event names can fail quietly. A plausible-looking PreCompact does nothing if it isn't in the known set. Comparing against a known-event list is the least interesting check in the script, and it's the one that actually caught me.

For how registration timing relates to firing conditions, I wrote up the move to declarative session definitions in the hooks that went from flaky to failing every time. Reading both makes it easier to see where each decision gets made.

I changed what I suspect first

I used to suspect the config whenever a hook didn't fire. Now I start with the hooks that do fire.

The range of problems caught at load time has genuinely widened. Still, not being rejected is not evidence of being correct. Counting how many tools a matcher actually hits was the one step that closed that gap.

Run your own hooks.json through both semantics once. Even with DEAD at zero, if the WIDE lines list tool names you don't recognize, that's where you're paying 31ms a call.