Treating Code Search as a Contract: Measuring the /codesearch Regex Default

The query that returned 4.5MB

The largest gap came from a query containing a delimiter.

As a literal, ja|en is just a string — a plausible way to write a path segment or a locale setting.

As a regex, it means ja or en. Against 979 Japanese MDX files it returned 26,658 lines: 4,564,685 bytes, about 4.5MB.

That fits in no model's context. An agent receiving it will truncate, summarize, or give up — and in every one of those cases it proceeds to the next step wearing the appearance of a successful search.

The literal form returned zero. Zero was the correct answer: the string ja|en does not exist in this repository.

Five characters. One reading gives 4.5MB, the other gives nothing. A single default decides which.

Parens stop you; dots don't

There's a second failure mode with a different character.

Looking for call sites, I tried getArticleContent(:

grep: Unmatched ( or \(

Read as a regex, that's an unclosed group. It errored out.

This is a good failure. It stops. You notice. Add -F and you get the 3 correct results.

The dangerous one doesn't stop. A dot raises no error, returns a plausible count, and slips 13 extra rows into the middle of it.

My unattended setup only ever watched the right-hand column. Paths that throw errors have retries and logging around them. Paths that drift quietly had nothing.

Errors invite design. Silence doesn't. I hadn't covered this, and there's no honest way to describe it other than a gap.

Writing the query as a contract

So I stopped treating queries as something you improvise, and started writing them as contracts. The same rules now live in the instructions I give agents.

1. Literal by default; regex only on purpose

If you're looking for a string, pass -F. Since forgetting the flag is the whole risk, I inverted the rule: use a regex only when the search genuinely requires one.

Collecting call sites across two functions requires one:

# Two call sites in one pass (a real use for regex)
/codesearch 'getArticleContent\(|getArticleBySlug\(' f:src/**/*.ts
 
# A plain string (literal is enough — and should be the default)
/codesearch -F 'antigravity.google' f:content/**/*.mdx

Notice the escaped \(. Needing an escape is evidence you meant a regex. Conversely, a regex query containing no escapes at all is almost always a missing -F.

That heuristic has been useful for auditing my own habits. Since adopting it I pause for a beat before hitting enter.

2. Close the search space with f: first

Narrowing the scope shrinks the absolute number of false positives — and brings the output back within range of a human glance.

Measuring premium: on the same repository:

About 51% disappeared — mostly build artifacts and the English versions, neither of which matters when auditing Japanese frontmatter.

Closing the scope isn't a performance move. It exists to keep the set you hand an agent small enough that a human can still sanity-check it. Nobody reads 1,934 lines. At 955 you can at least judge whether the count is plausible.

3. Decide the expected magnitude in advance

Before running a search, estimate roughly how many hits it should return. If the order of magnitude is off, the query is broken.

That rule alone would have stopped the 26,658 lines instantly. If you're hunting for a locale string and get five digits, you're not hunting for what you think you are.

The agent instruction reads: "If results exceed the expected count by 10x, stop and report the query instead of proceeding." The bound doesn't need to be precise. Catching the order of magnitude is enough.

A small harness for measuring drift

Rules are one thing; compliance is another. So I keep a script that periodically measures the regex-vs-literal gap for the queries I actually use.

#!/usr/bin/env python3
"""codesearch_drift.py — measure the gap between the regex default and literal matching.
 
/codesearch reads queries as regular expressions by default. This reproduces
those semantics with ripgrep and flags queries whose drift exceeds a threshold.
 
Usage:
    python3 codesearch_drift.py <repo_root> queries.txt
"""
import subprocess
import sys
from pathlib import Path
 
# Warn above this share of excess hits
DRIFT_THRESHOLD = 0.05
# Beyond this magnitude, the query is probably broken
VOLUME_CEILING = 5000
 
 
def count(root: Path, query: str, literal: bool) -> int:
    """Return matching line count. rg exits 1 on zero matches, so map that to 0."""
    cmd = ["rg", "--count-matches", "--no-messages"]
    if literal:
        cmd.append("--fixed-strings")
    cmd += [query, str(root)]
    proc = subprocess.run(cmd, capture_output=True, text=True)
    # Do not pipe this: a pipe would swallow the exit code we depend on.
    if proc.returncode == 1:
        return 0
    if proc.returncode >= 2:
        return -1  # regex syntax error (the failure that halts)
    return sum(int(l.rsplit(":", 1)[1]) for l in proc.stdout.splitlines() if ":" in l)
 
 
def audit(root: Path, queries: list[str]) -> int:
    problems = 0
    for q in queries:
        re_hits = count(root, q, literal=False)
        fx_hits = count(root, q, literal=True)
 
        if re_hits == -1:
            print(f"SYNTAX  {q!r}: invalid as a regex (should use -F)")
            problems += 1
            continue
 
        if re_hits > VOLUME_CEILING:
            print(f"VOLUME  {q!r}: {re_hits:,} lines exceeds ceiling {VOLUME_CEILING:,}")
            problems += 1
 
        if fx_hits == 0 and re_hits > 0:
            print(f"INVERT  {q!r}: literal 0 / regex {re_hits:,} lines — meaning is inverted")
            problems += 1
            continue
 
        if fx_hits > 0:
            drift = (re_hits - fx_hits) / fx_hits
            if drift > DRIFT_THRESHOLD:
                print(f"DRIFT   {q!r}: regex {re_hits} / literal {fx_hits} (+{drift * 100:.1f}%)")
                problems += 1
 
    return problems
 
 
def main() -> int:
    if len(sys.argv) != 3:
        print(__doc__)
        return 2
    root = Path(sys.argv[1])
    queries = [
        line.strip()
        for line in Path(sys.argv[2]).read_text(encoding="utf-8").splitlines()
        if line.strip() and not line.startswith("#")
    ]
    problems = audit(root, queries)
    print(f"\nChecked {len(queries)} queries / {problems} need attention")
    return 1 if problems else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

queries.txt holds the searches you reach for daily:

# Auditing outbound links
antigravity.google
# Checking frontmatter
premium:
# Locale notation (this one trips INVERT)
ja|en

Run against this repository, antigravity.google raises DRIFT and ja|en raises INVERT. Both were queries I had genuinely been running.

One implementation note. I initially piped rg output through head, which swallowed the exit code for syntax errors — so every broken query reported as healthy. Never pipe a command whose exit code you depend on. A harness that fails silently defeats its own purpose.

What belongs in the agent instructions

With rules and a harness in place, the last step is telling the agent. Three lines:

1. Pass -F when searching for a plain string. A regex query with no escapes counts as a missing flag.
2. Always close the scope with f:. Never target the whole repository.
3. If the hit count exceeds the expected magnitude, report the count and query instead of proceeding.

I resisted the urge to write more. Agent instructions get followed less the longer they get.

The same thought surfaced reading another 1.1.3 fix — the one where headless runs (-p) had been silently auto-approving tools that should have asked, now changed to a soft denial that names the required allow rule on stderr.

Where the default doesn't lean toward safety, no amount of instruction text closes the gap. Where the default is safe, short instructions suffice.

The /codesearch regex default isn't a dangerous default. It's a powerful one. But a powerful default is only powerful in the hands of someone who knows it's there.

Where to start

Write down ten queries you run regularly. Check whether any contain a filename with a dot, or a |. If they do, run the harness above and the gap shows up as a number.

Two of my ten did. I'd been running them for the better part of a year without noticing.

The more you delegate to an agent, the more the step before the delegation matters. Search is that first step.