Three Quarters of My Reference Notes Never Reached the Agent: Measuring What head Cuts Away

head never fails, so the loss is invisible

This is the part that bothered me most. Nothing in this path reports a failure.

The exit status of cat a | head -150 belongs to the last command in the pipeline. head succeeds on empty input. So a typo in a path, a vanished file, and a mid-sentence truncation all flow downstream wearing the same face: a clean exit with nothing, or almost nothing, behind it.

The agent will not report the gap either. Information that never arrived does not exist from its point of view. What looked like "it ignored the note" was really "the note was never handed over."

In unattended runs this kind of silence is expensive. I have closed a similar path before, where agent output disappeared for an unrelated reason; that one is written up in the pre-commit gate for agent output swallowed by .gitignore. The shape is identical: unless a failure raises its hand, a broken setup runs happily for weeks.

Make the boundary a section-level contract

The fix followed from the diagnosis. Cut by section instead of by line, budget in bytes, and always break immediately before a heading. Sections that do not fit are dropped whole rather than sliced.

The notes are Markdown with ## section headings, so the packer is small.

#!/usr/bin/env python3
"""notes_pack.py — pack reference notes into a byte budget, section by section"""
import re
import sys
from pathlib import Path
 
HEADING = re.compile(r"^##\s+(.+)$")
 
 
def split_sections(text: str):
    """Split into a preamble plus one entry per ## section."""
    lines = text.splitlines(keepends=True)
    preamble, sections = [], []
    current = None
    for line in lines:
        m = HEADING.match(line)
        if m:
            if current:
                sections.append(current)
            current = {"title": m.group(1).strip(), "body": [line]}
        elif current:
            current["body"].append(line)
        else:
            preamble.append(line)
    if current:
        sections.append(current)
    for s in sections:
        s["text"] = "".join(s["body"])
        s["bytes"] = len(s["text"].encode("utf-8"))
    return "".join(preamble), sections
 
 
def pack(path: Path, budget: int, keep_titles=()):
    text = path.read_text(encoding="utf-8")
    preamble, sections = split_sections(text)
    out, used, dropped = [], len(preamble.encode("utf-8")), []
 
    # (1) reserve sections that are explicitly required
    pinned = [s for s in sections if any(k in s["title"] for k in keep_titles)]
    rest = [s for s in sections if s not in pinned]
 
    for s in pinned + rest:
        if used + s["bytes"] <= budget:
            out.append(s)
            used += s["bytes"]
        else:
            dropped.append(s["title"])  # (2) drop whole sections, never slice
 
    body = preamble + "".join(s["text"] for s in out)
    return body, used, dropped
 
 
if __name__ == "__main__":
    src = Path(sys.argv[1])
    budget = int(sys.argv[2]) if len(sys.argv) > 2 else 20000
    keep = sys.argv[3].split(",") if len(sys.argv) > 3 else []
    body, used, dropped = pack(src, budget, keep)
    sys.stdout.write(body)
    # (3) never let a drop happen quietly
    if dropped:
        sys.stderr.write(
            f"[notes_pack] {src.name}: {used}/{budget} bytes, "
            f"dropped {len(dropped)} sections: {', '.join(dropped[:5])}\n"
        )

Three things carry the design.

1. Required sections are reserved first. A section like "device-specific branches" is needed regardless of how new it is, so keep_titles claims it before anything else competes for the remaining budget.
2. Sections that do not fit are dropped whole. A fragment is worse than an absence, because a fragment persuades the agent that the topic was considered and no conclusion was reached.
3. Every drop goes to stderr. It is the smallest possible mechanism for refusing to lose things quietly.

The only real difference from head is that the cut now sits where a human decided meaning ends. It is not clever. It still changed the quality of what arrives.

Deciding what survives the budget

The moment you drop whole sections, someone has to choose which ones. I could not automate that, so I moved the decision into the notes themselves.

<!-- notes-priority: pinned -->
## Device-specific branches (needed every run)
 
<!-- notes-priority: fresh -->
## Update, 2026-07-13

The packer reserves pinned sections, fills fresh ones newest-first, and leaves undeclared sections for last.

I set the budget itself against the instruction, not against the model's window. My rule of thumb: if the reference material exceeds roughly three times the length of the instruction, tighten the budget. When reference overwhelms instruction, the agent drifts toward summarizing the material and quietly demotes the job I asked for. The same dilution shows up with long attachments, which is what the grounding gate for long PDFs the agent skims past addresses from the reading side. Seen from the handing-over side, the first defense is simply a budget that refuses to overfill.

A preflight that catches the gap

Even with the packer, a mistyped path and an empty file still slip through in silence. So I added a gate that runs before anything launches.

#!/usr/bin/env bash
# notes_preflight.sh — verify notes exist, are non-empty, and carry required sections
set -euo pipefail
 
NOTES="${1:?usage: notes_preflight.sh <notes-dir>}"
FAIL=0
 
check() {
  local file="$1"; shift
  local path="${NOTES}/${file}"
 
  if [ ! -f "$path" ]; then
    echo "MISSING notes file: ${path}"; FAIL=$((FAIL + 1)); return
  fi
  if [ ! -s "$path" ]; then
    echo "EMPTY notes file: ${path}"; FAIL=$((FAIL + 1)); return
  fi
  # required sections: a file without these is considered broken
  for key in "$@"; do
    if ! grep -qF "$key" "$path"; then
      echo "MISSING section: ${path} -> ${key}"; FAIL=$((FAIL + 1))
    fi
  done
  echo "OK ${file} ($(wc -c < "$path") bytes)"
}
 
check "release_steps.md" "## Device-specific branches" "## Rollback"
check "known_issues.md"  "## Open"
 
if [ "$FAIL" -gt 0 ]; then
  echo "STOP: ${FAIL} notes problem(s). Not launching the agent."
  exit 1
fi

The order in the unattended run is now fixed.

1. Run notes_preflight.sh first and do not launch the agent if anything is off. Doing nothing today beats working from broken material.
2. Run notes_pack.py per file, keeping stderr in the run log.
3. Assemble the prompt and start the agent.

Refusing to proceed on a broken premise is the same instinct that governs the schedule as a whole; where to hand work over and where to stop is something I worked through in how I split scheduled work after moving to the Antigravity CLI. This preflight simply adds one more stop condition, this time on the material rather than the task.

What three weeks showed, and what stayed broken

I watched the morning runs for about three weeks after the switch.

The reassuring part was not the percentages. It was that runs stopped walking into workarounds I had already written down. Once the cause turned out to be my own handover, I spent far less time second-guessing the agent's output.

To be honest about the limits: sections that exceed the budget still do not arrive. Dropping them whole is a choice of absence over fragments, not a way of delivering more. I read the stderr drops and decide by hand when to fold the notes down. I tried routing that to an automatic summarizer, but the summaries kept losing the caveats that made a section worth keeping, so for now I prefer doing it myself.

One more limit: this design depends on the notes being structured with headings. A file of endless running bullets has no section boundaries, and the packer can do nothing with it. The work of structuring notes stays on my side of the table.

The one thing to do next

If you hand reference material to an agent anywhere in your setup, measure the reach before changing anything. The for loop at the top of this article tells you, in about a minute, what share you were actually delivering.

For me, that one number changed who I suspected. If it does the same for you, the rest of the design tends to follow on its own.