The File Counts Matched. 148 Pairs Disagreed on Code Blocks

A length ratio looked cheap, and it did not work

My first instinct was the cheapest possible rule: divide the English body length by the Japanese one and flag anything far from parity. A few lines of code.

Then I plotted the ratio across all 1,057 pairs, and the premise turned out to be wrong.

Japanese carries more meaning per character, so a faithful English rendering of the same content runs about 1.5 times longer. The center of the healthy distribution sits at 1.56, not at 1.0.

Draw a band of plus or minus 30% around 1.0 and only 134 of 1,057 pairs fall inside it. The other 87% become warnings. A list like that gets ignored within a week, and I would have been the one ignoring it.

Recentering the band on 1.56 does not rescue the idea either. The pair that lost 25 code blocks has a ratio of 0.65. It sits low, but so do plenty of pairs whose English rendering is simply more compact. Volume cannot separate an omission from a tighter paragraph.

A ratio of quantities is not evidence of an omission. The only things worth counting are the ones translation should leave untouched. I widened and narrowed that band three times before I accepted this, and none of those attempts found anything.

The direction of the drift was the opposite of what I expected

My second assumption was wrong too.

I had pictured the English side thinning out — content lost somewhere in translation. Counting the direction told a different story.

The majority run the other way: the English version has grown. Nothing was lost. One side kept receiving edits that the other side never got, and over enough edits the two documents became two different articles.

I recognize how it happened. I fix something in Japanese and forget the counterpart. Or I reread the English page, decide an explanation is thin, and add a section there only. Each decision is small and reasonable. Repeat it across 1,057 pairs and a translation pair turns into a fork.

Editing one side at a time does not degrade a translation. It forks it. Degradation is visible. A fork reads fine on both sides, which is exactly why nobody notices.

Forty lines are enough to check it

Once the list of countable things is settled, the implementation is short. This is the whole file, sitting at the repository root. It does not fix anything — it finds and prints.

#!/usr/bin/env python3
"""Structural parity check for JA/EN article pairs. Prints only pairs past the tolerance."""
import os
import re
import sys
 
ROOT = "content/articles"
FENCE = "`" * 3  # the fence marker itself
FENCE_TOL = 3  # allowed difference in code block count
H2_TOL = 3     # allowed difference in H2 count
 
 
def body_of(path):
    text = open(path, encoding="utf-8").read()
    m = re.match(r"^---\n.*?\n---\n(.*)$", text, re.S)
    return m.group(1) if m else text
 
 
def shape(body):
    # Fences appear twice per block, once opening and once closing
    return {
        "fence": body.count(FENCE) // 2,
        "h2": len(re.findall(r"^## ", body, re.M)),
        "table": body.count("<table"),
    }
 
 
def main():
    findings = []
    for dirpath, _dirs, files in os.walk(os.path.join(ROOT, "ja")):
        for name in sorted(files):
            if not name.endswith(".mdx"):
                continue
            ja = os.path.join(dirpath, name)
            en = ja.replace(f"{ROOT}/ja/", f"{ROOT}/en/", 1)
            if not os.path.exists(en):
                findings.append((ja, "no English counterpart"))
                continue
            a, b = shape(body_of(ja)), shape(body_of(en))
            if a["fence"] > 0 and b["fence"] == 0:
                findings.append((ja, f"English has no code at all {a['fence']} -> 0"))
            elif abs(a["fence"] - b["fence"]) >= FENCE_TOL:
                findings.append((ja, f"code blocks {a['fence']} -> {b['fence']}"))
            if abs(a["h2"] - b["h2"]) >= H2_TOL:
                findings.append((ja, f"H2 headings {a['h2']} -> {b['h2']}"))
    for path, reason in findings:
        print(f"{path}: {reason}")
    print(f"-- {len(findings)} findings", file=sys.stderr)
    return 1 if findings else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

The division by two deserves a word. Each block contributes an opening fence and a closing one, so without the division every count doubles. An unclosed fence leaves a remainder of one, and the floor division hides it, so if you also want to catch unbalanced fences, add a line that looks at that remainder. My Markdown formatter rejects unclosed fences already, so I left it out.

I count <table> but do not gate on it. Tables get added to one language on purpose often enough, and no reader has been hurt by it yet. Counting something without judging it is what keeps the report short enough to keep reading.

Why the tolerance is three

The number of findings swings hard with the tolerance.

At a tolerance of one, the report opens with 148 and 196 entries. That is a report you close before reading. At three, after removing pairs counted twice, 80 pairs remain — 7.6% of the tree. A few a day and the bottom is in sight within the month.

I work through them in three tiers.

Pairs where the English side has zero code

Four of them. The page does not function for the reader, so these get fixed the same day. I treat zero as its own category rather than as a large difference.

Pairs differing by three or more

Eighty pairs. These need eyes, because the fix depends on which way it drifted. Restoring something dropped and folding an extra section back into the pair are different jobs, and I decide per pair.

Pairs differing by one or two

The remainder. Mostly an added table, or one section split in two on one side. These wait until I have another reason to touch the article. Chasing all of them turns tidying into the goal.

Hand the agent a countable condition, not an instruction

For a while I asked Antigravity to "apply the same change to the English version." The results were not good. Some days it happened, some days it did not, and afterwards I could not tell which.

I ask differently now. The agent handles the prose; the script decides whether the pair is aligned. My AGENTS.md carries one line: after editing a Japanese article, run the structural parity check and stop with a report if the output is not empty. The check itself is the script above, and the agent only runs it and reads the result.

Since 2.11.0, Antigravity can discover custom skills and rules from configuration files placed in project subdirectories. Only my bilingual repositories need this convention, so instead of piling the rule onto one top-level AGENTS.md, I keep it next to the directory it governs. The larger a rules file grows, the harder it becomes — for me, not for the agent — to remember which rule is in effect.

Before a push, it runs only when the change actually touches MDX.

#!/usr/bin/env bash
# .git/hooks/pre-push — check structural parity only when MDX changed
set -euo pipefail
 
if ! git diff --cached --name-only HEAD 2>/dev/null | grep -q '\.mdx$'; then
  # Nothing in the outgoing commits touches MDX, so pass quietly
  if ! git diff --name-only origin/main...HEAD | grep -q '\.mdx$'; then
    exit 0
  fi
fi
 
if ! python3 tools/pair_shape_check.py; then
  echo "Structural parity differs. Read the list above before pushing." >&2
  exit 1
fi

The one boundary I have not moved is that nothing gets repaired automatically. Ask an agent to fill in what the English version is missing and it writes explanations the Japanese version never had. You set out to close a fork and you open a new one.

Translation is the agent's work. Counting is mine. I keep that line on the days I am in a hurry, which are the days it matters. If you want the neighboring failure modes, I wrote about frontmatter values being rewritten without an error in When YAML frontmatter rewrites your values without raising an error, and about tracked files dropping out of agent search in Git Tracks the File, but Your Agent Can't See It. Both belong to the same family: things that look like they passed.

Where I would start counting

If your repository has the same shape, I would run one line before worrying about tolerances or headings at all. It lists only the pairs where the source language has code and the other language has none.

python3 - <<'PY'
import os, re
R = "content/articles"
F = "`" * 3
for d, _, fs in os.walk(f"{R}/ja"):
    for f in fs:
        if not f.endswith(".mdx"):
            continue
        ja = os.path.join(d, f)
        en = ja.replace(f"{R}/ja/", f"{R}/en/", 1)
        if not os.path.exists(en):
            continue
        cnt = lambda p: open(p, encoding="utf-8").read().count(F) // 2
        if cnt(ja) > 0 and cnt(en) == 0:
            print(ja, cnt(ja), "-> 0")
PY

In my case the four pairs it printed were the four that had been failing readers the longest. If it prints nothing, there is nothing to do today. If it prints something, that is one article you can repair before the day ends.

Thank you for reading this far. If you also maintain a repository in two languages, I hope this gives you somewhere to start counting.