Keeping Naming and Formatting Stable When Your Model Falls Back

Fixing the style contract with a minimal setup

First, put down the configuration that folds any model's output into one shape. The assumption is that this runs immediately after the agent's generation step.

// .prettierrc.json — formatting decided uniquely here
{
  "semi": true,
  "singleQuote": true,
  "tabWidth": 2,
  "useTabs": false,
  "trailingComma": "all",
  "importOrder": ["^react", "^@/", "^\\."],
  "importOrderSeparation": true
}

Naming is judged mechanically by the linter. ESLint's @typescript-eslint/naming-convention reliably catches snake_case even when the model mixes it in.

// eslint.config.js (excerpt) — stop naming drift mechanically
export default [
  {
    rules: {
      '@typescript-eslint/naming-convention': [
        'error',
        { selector: 'function', format: ['camelCase'] },
        { selector: 'variable', format: ['camelCase', 'UPPER_CASE'] },
        {
          selector: 'variable',
          types: ['boolean'],
          format: ['PascalCase'],
          prefix: ['is', 'has', 'should', 'can'],
        },
      ],
    },
  },
];

That covers formatting and naming. In the agent workflow I wedge four fixed steps: generate, then Prettier, then ESLint --fix, then an ESLint check. Auto-fixable violations get fixed automatically, and only the un-fixable ones reach a human.

Detecting in-run drift from the diff

The Layer 3 drift probe is the center of this design. The goal is to catch cases that pass the linter but where habits changed between the first and second half of the run — for example, a style that started with early returns shifting to nested if blocks.

Across the files a single agent run touched, measure a few simple metrics and compare the first half against the second.

# drift_probe.py — surface where writing habits drifted within a run
import re, subprocess, sys
 
def metrics(path: str) -> dict:
    src = open(path, encoding="utf-8").read()
    lines = src.splitlines()
    return {
        "early_return": len(re.findall(r"\breturn\b", src)),
        "nested_if": sum(1 for l in lines if re.match(r"\s{6,}if\b", l)),
        "snake_ident": len(re.findall(r"\b[a-z]+_[a-z]+\b", src)),
        "arrow_fn": len(re.findall(r"=>", src)),
        "loc": len(lines),
    }
 
def normalize(m: dict) -> dict:
    loc = max(m["loc"], 1)
    return {k: round(v / loc, 4) for k, v in m.items() if k != "loc"}
 
def changed_files() -> list:
    out = subprocess.check_output(
        ["git", "diff", "--name-only", "HEAD~1", "HEAD"], text=True
    )
    return [f for f in out.splitlines() if f.endswith((".ts", ".tsx"))]
 
files = changed_files()
half = len(files) // 2
early = [normalize(metrics(f)) for f in files[:half]]
late = [normalize(metrics(f)) for f in files[half:]]
 
def avg(rows, key):
    return sum(r[key] for r in rows) / max(len(rows), 1)
 
drift = {}
for key in ("nested_if", "snake_ident", "arrow_fn"):
    a, b = avg(early, key), avg(late, key)
    if a and abs(b - a) / a > 0.35:  # treat >35% change as drift
        drift[key] = (round(a, 3), round(b, 3))
 
if drift:
    print("⚠️ style drift detected:", drift)
    sys.exit(1)
print("✅ consistent within run")

The 35% threshold is the value that settled after running for a few weeks across my four app repositories. At 10% it fired on normal file variation; at 50% it missed real drift. Tune it per project. Copying it without a basis of your own gives you an alarm that fires so often nobody looks at it.

What tripped me up in production, and the fixes

The first pitfall was not measuring the fallback itself. When drift appeared, I could not tell whether it came from a model swap or simply from the task's nature changing. So I started recording whether a fallback occurred as run metadata and cross-referencing it with the drift result. If the drift rate only spikes on runs where a fallback happened, the cause is the model.

The second pitfall was over-trusting --fix. Prettier and ESLint auto-fix repair formatting, but the fix can occasionally fold in a semantic change as a side effect. Splitting the auto-fix commit from the generation commit — keeping fixes in a separate commit — lets you separate "real change" from "reformatting" at review time. That paid off during diagnosis.

The third was how to operate the threshold. Wiring the drift probe to exit 1 as a merge gate from day one stalls development during the tuning period. For the first two weeks I kept it warning-only (log via exit 0), locked the threshold on real data, and only then turned it into a gate. Standing up a gate before you measure is, in my experience, the most resented way to roll something out.

Where machines decide, and where humans look

My recommendation is to hand formatting and naming 100% to machines. Consistency is all that matters there, and it is not worth a human's deliberation. Structural drift — function splitting, early-return preference — I detect but deliberately do not auto-fix, because structure depends on context and a blanket machine fix can hurt readability instead.

This line mattered most when touching initialization code tied directly to AdMob revenue, where a break shows up in sales immediately. When formatting noise inflates a diff, you miss the one line that actually deserves attention. Erase the noise a machine can erase, and reserve human focus for meaningful diffs. That is the quietest, most dependable foundation I have found for holding quality steady no matter how often the model swaps underneath you.

Running several Dolice Labs sites solo, how fast I can put a number on these quietly-costly phenomena is what decides whether I can keep going. Fallback-induced drift was a textbook case. Measure first, fold with machines, then look at what remains. I intend to keep that order intact in the next repository I touch.