Reading the Same History in 38 Seconds — or 0.4: Handing a Read-Only .git to Your Agent

The Obvious Loop Took 38 Seconds

I measured against synthetic repositories rather than a real one, because I wanted to vary exactly one thing: commit count. A generator streams commits into git fast-import — 60 modules, implementation and test files co-changing 72% of the time, a shared registry dragged in 18% of the time, all under a fixed random seed. Generating 8,000 commits takes 2.2 seconds, which makes redoing an experiment painless.

# gen_fixture.py (excerpt): build synthetic history via a fast-import stream
files = [f"src/api/mod_{m}.py"]
if rng.random() < 0.72:
    files.append(f"tests/test_mod_{m}.py")   # impl/test co-change
if rng.random() < 0.18:
    files.append("src/core/registry.py")     # shared-file entanglement
# ... assemble blobs and commits as text, then feed the stream once
subprocess.run(["git", "-C", path, "fast-import", "--quiet"],
               input=stream, check=True)

The first implementation is the one anybody writes first. Get the commit list from git rev-list, then ask each commit for its changed files with git diff-tree and its author with git show -s.

# The obvious implementation: one git invocation per commit
shas = run(["rev-list", "HEAD"], repo).split()
for sha in shas:
    out = subprocess.run(
        ["git", "-C", repo, "diff-tree",
         "--no-commit-id", "--name-status", "-r", sha],
        capture_output=True, text=True, check=True).stdout
    files = [l.split("\t", 1)[1] for l in out.splitlines() if "\t" in l]
    churn.update(files)
    # author fetched the same way, via git show -s --format=%an

It works. The output is correct. It also takes 38.4 seconds at 8,000 commits.

Adding 38 seconds to every session start was never going to survive. That option died here.

The Same Output From a Single git Call

The second implementation folds everything into one invocation. git log --name-status emits every commit's changed files and author in a single run, and a single pass over that output rebuilds the same aggregates.

# One-pass implementation: git starts exactly once
def build_onepass(repo):
    out = run(["log", "--name-status", "--pretty=format:@%H|%an"], repo)
    churn, pairs, authors = Counter(), Counter(), Counter()
    files, n = [], 0
    def flush():
        churn.update(files)
        for a, b in itertools.combinations(sorted(files)[:10], 2):
            pairs[(a, b)] += 1
    for line in out.splitlines():
        if line.startswith("@"):          # commit boundary
            if n:
                flush()
            files, n = [], n + 1
            authors[line.split("|", 1)[1]] += 1
        elif "\t" in line:                # a name-status line
            files.append(line.split("\t", 1)[1])
    if files or n:
        flush()
    return pack(churn, pairs, authors, n)

Co-change pairs are capped at the first 10 files per commit. The occasional bulk commit — a repository-wide reformat, say — would otherwise explode the pair count quadratically and let a single commit dominate the statistics.

The results, after confirming both implementations agree on the top hotspots and top co-change pairs:

Each value is the median of three runs (the 8,000-commit per-commit loop ran once, for time reasons). Environment: Linux 6.8, 4 vCPUs, 3.8 GB RAM, git 2.34.1, Python 3.10.12. These numbers are environment-dependent — rerun the scripts on your own machine before relying on them.

The Cost Was Never the Data. It Was Process Startup

My prediction going in was the opposite. I expected the single git log to be the risky one: asking one process to dump name-status output for 8,000 commits sounds expensive, and I worried about the memory footprint of catching it all.

What was actually expensive was starting processes. Divide the per-commit loop's time by the number of commits and a pattern appears:

Roughly 4.7 ms per commit, nearly flat across repository sizes. The loop launches two subprocesses per commit — diff-tree and show — so each git process costs about 2.3 to 2.4 ms in startup and teardown alone. The time was disappearing at the doorway, not inside the work.

On its own, 2.3 ms is invisible. Multiplied by two spawns across 8,000 commits, it becomes 38 seconds. The one-pass version receives the same volume of information and finishes in 0.38 seconds; a plain Python parsing loop was more than fast enough.

The lesson transfers directly to agent tool design. Build a tool that returns information about one commit, and an agent that diligently loops over it pays full spawn cost on every iteration. History aggregation belongs inside the tool, folded into a single git invocation, returning only the finished summary.

Shallow Clones Corrupt the Pack Silently

Back to this morning. What happens when this pack is generated against a --depth 1 copy?

No error. The pack builds successfully. Only the contents are meaningless.

In my measurement, cloning the 2,000-commit repository with --depth 1 left git rev-list --count HEAD returning 1, and the pack's top hotspot became "docs/changelog.md, changed once" — whichever files happened to ride the final commit, crowned as the most-churned code in the project.

The silence is the dangerous part. The aggregation succeeds, the JSON is well-formed, and the agent reads it in good faith: "the most frequently changed file in this repository is docs/changelog.md." A false context is worse than no context.

The fix is blunt: verify the history actually exists before building anything, and fail closed if it does not.

def assert_history(repo, min_commits=50):
    count = int(run(["rev-list", "--count", "HEAD"], repo))
    shallow = os.path.exists(os.path.join(repo, ".git", "shallow"))
    if shallow or count < min_commits:
        raise SystemExit(
            f"history too short: commits={count} shallow={shallow} — "
            "fetch full history before building the pack")

The .git/shallow check exists because a commit-count threshold alone cannot distinguish a genuinely young repository from a truncated one. Provisioning for automated environments defaults to depth-1 clones more often than not, so if you build anything on read-only .git, I would put this guard in on day one.

Settling Into a Routine

With the numbers in hand, the setup I landed on is simple.

The pack is rebuilt at every session start. At 0.38 seconds and 1.7 KB for 8,000 commits, it vanishes into the startup noise. The output goes into the working directory as a file — not inlined into the prompt — and sits among the first things the agent reads.

The exclusions are worth recording too. Full diff bodies never get passed; they are large, and their presence defeats the point of a summary. Whole-repository blame sweeps are also out of this pack. Hotspots and co-change already answer "where should I be careful," and line-level provenance can be fetched on demand for the specific file that needs it.

Read-only .git is not a flashy change. But being able to implement "pass the information, withhold the authority" plainly, without workarounds, feels like solid ground gained.

Start by running git rev-list --count HEAD on the repository your agent actually sees. Whether that number matches your expectation is the first real step toward handing history to an agent — and if the structure of these scripts saves you a detour, I am glad for it.