The Thumbnails Had Been Drifting: Catching Image-Asset Regressions with pHash and SSIM in Your Agent Workflow

What pHash misses and SSIM catches

But the banding that troubled me slipped right past pHash. Tonal stair-stepping is local and barely moves the low-frequency DCT components, so the Hamming distance stayed at 1 while the look had clearly degraded. This is SSIM's moment.

SSIM (Structural Similarity Index) scans two images with a small sliding window, comparing luminance, contrast, and structure locally, then rolls that into a score from 0 to 1. 1.0 is a perfect match, and it drifts down as local noise or crushed gradients appear. scikit-image has an implementation.

import numpy as np
from PIL import Image
from skimage.metrics import structural_similarity as ssim
 
def ssim_score(path_a: str, path_b: str) -> float:
    """SSIM between two images. 1.0 = perfect match."""
    img_a = Image.open(path_a).convert("L")
    img_b = Image.open(path_b).convert("L")
    # SSIM assumes identical dimensions; conform the candidate to the baseline
    if img_b.size != img_a.size:
        img_b = img_b.resize(img_a.size, Image.LANCZOS)
    arr_a = np.asarray(img_a)
    arr_b = np.asarray(img_b)
    return float(ssim(arr_a, arr_b))
 
if __name__ == "__main__":
    s = ssim_score("baseline/thumb_001.jpg", "candidate/thumb_001.jpg")
    print(f"SSIM = {s:.4f}")

The key detail is that SSIM assumes identical dimensions. Mismatched sizes raise an exception, so you resize the candidate to the baseline's dimensions first. If you want to include color, drop the grayscale conversion and pass RGB arrays with ssim(arr_a, arr_b, channel_axis=2). That makes it sensitive to color-profile differences, though, so I prefer to run grayscale first and measure color drift separately.

Measuring that banding with SSIM, the score had fallen to 0.983. What pHash saw as "identical" at distance 1, SSIM surfaced clearly as degradation. That difference is the reason to use both.

Setting thresholds from measurement, not intuition

Declaring "distance 5 or under is fine" out of thin air is risky. Image characteristics differ per app. Whether your assets are photographic, gradient-heavy, or contain text changes how much the scores move even for healthy edits.

My procedure is simple. First, run a large batch of re-encodes with no intentional change and measure the distribution of pHash distance and SSIM. Then mix in deliberately degraded images — lower the quality, over-sharpen, swap the subject — and find the boundary where the two populations separate. Add a safety margin to that boundary and use it as your threshold.

Here's what I measured on my own wallpaper assets. These are guides from my data only; always re-measure on yours.

The important thing is not to start from "fail (block)." For the first two weeks, keep everything at "warn" and observe how often, and on which kinds of changes, the scores actually move. Only after the lay of the land is visible should you draw the blocking line. I ignored this order once, went red on every legitimate design update, and had to disable the gate entirely for a while.

Rolling it into a gate script

With both rulers in hand, you compare a baseline directory against a candidate directory and report only the images that deviate. The trick is to emit an exit code an agent or CI can read, plus a diff report for later human review.

#!/usr/bin/env python3
"""image_gate.py — compare baseline and candidate image assets perceptually."""
import sys
import json
from pathlib import Path
import numpy as np
from PIL import Image
import imagehash
from skimage.metrics import structural_similarity as ssim
 
# Thresholds (re-measure on your own data)
PHASH_WARN, PHASH_FAIL = 3, 7
SSIM_WARN, SSIM_FAIL = 0.99, 0.95
 
def compare(base: Path, cand: Path) -> dict:
    a, b = Image.open(base), Image.open(cand)
    dist = imagehash.phash(a) - imagehash.phash(b)
    ga = np.asarray(a.convert("L"))
    gb_img = b.convert("L")
    if gb_img.size != a.size:
        gb_img = gb_img.resize(a.size, Image.LANCZOS)
    score = float(ssim(ga, np.asarray(gb_img)))
    if dist >= PHASH_FAIL or score < SSIM_FAIL:
        verdict = "fail"
    elif dist >= PHASH_WARN or score < SSIM_WARN:
        verdict = "warn"
    else:
        verdict = "pass"
    return {"name": base.name, "phash": dist, "ssim": round(score, 4), "verdict": verdict}
 
def main(baseline_dir: str, candidate_dir: str) -> int:
    base_root, cand_root = Path(baseline_dir), Path(candidate_dir)
    results, worst = [], "pass"
    for base in sorted(base_root.glob("*.jpg")):
        cand = cand_root / base.name
        if not cand.exists():
            results.append({"name": base.name, "verdict": "missing"})
            worst = "fail"
            continue
        r = compare(base, cand)
        results.append(r)
        if r["verdict"] == "fail":
            worst = "fail"
        elif r["verdict"] == "warn" and worst != "fail":
            worst = "warn"
    Path("image_gate_report.json").write_text(json.dumps(results, indent=2, ensure_ascii=False))
    for r in results:
        print(f"{r['verdict']:>7}  {r['name']}  phash={r.get('phash','-')}  ssim={r.get('ssim','-')}")
    print(f"\n=== overall: {worst} ===")
    return 1 if worst == "fail" else 0
 
if __name__ == "__main__":
    sys.exit(main(sys.argv[1], sys.argv[2]))

The script returns exit code 1 if there's a single fail; a warn stays at code 0 and defers to a human. Keeping image_gate_report.json lets you later hand it to an agent to read which image tripped and why, so it can localize the cause. Treating a missing file (missing) as fail is deliberate: it keeps a pipeline that quietly forgets to emit some images from slipping through.

Wiring it into your Antigravity workflow

Everything so far has value run by hand. But in a workflow where you delegate implementation to agents — as with Antigravity — it pays to close the loop so the agent that made the change runs the gate and fixes its own red.

I keep this passage in AGENTS.md:

## Rules for changing the image pipeline
 
If you touch `scripts/generate_thumbnails.py` or any Pillow resampling /
quality setting, you must run the following after the change:
 
    python scripts/generate_thumbnails.py --out candidate/
    python image_gate.py baseline/ candidate/
 
When overall is `fail`, do not produce a merge diff. Read
image_gate_report.json, explain the cause, and revisit the generation-side
settings — not the thresholds. Never update baseline/ without human approval.

That last sentence matters most. When the gate is red, agents often take the shortcut of "update the baseline so it passes." Unless you forbid it, they'll repaint degradation as normal. Baseline updates should happen only when a human confirms an intentional design change — putting that boundary into words is what keeps the automation safe.

In CI, layer it lightly onto GitHub Actions.

name: image-asset-gate
on: [pull_request]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pillow imagehash scikit-image numpy
      - run: python scripts/generate_thumbnails.py --out candidate/
      - run: python image_gate.py baseline/ candidate/
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: image-gate-report
          path: image_gate_report.json

if: always() keeps the report as an artifact even when the gate fails — and failure is exactly when the report becomes the clue. If you want broader visual regression across parallel agents, pairing this with how to design parallel-agent visual regression covers both web UI and asset images.

Where people stumble

EXIF orientation. Images from phones sometimes encode rotation in the EXIF Orientation tag. Image.open doesn't apply it automatically, so baseline and candidate can disagree on orientation and pHash spikes. Normalize with ImageOps.exif_transpose(img) before comparing.

Alpha channel. Running convert("L") on a transparent PNG crushes the transparent regions to black and rattles SSIM. Composite assets with transparency onto a known background color before comparing for stable scores.

Resize order for mismatched sizes. When you resize the candidate for SSIM, upscaling and downscaling favor different algorithms. I standardize on LANCZOS for thumbnail comparison; changing it changes the meaning of your thresholds.

Intentional design changes. A refreshed logo or a new palette will turn the gate red even though the change is legitimate. That's not a bug — it's the design. Provide an explicit human step (say, make update-baseline) and keep that action under review, so the intent is recorded in history. If you handle resolution variants as a batch, connecting this with designing wallpaper resolution-variant delivery makes it easier to decide which dimensions the gate should cover.

Wrapping up — stand up a "warn-only" gate this week

Visual regressions are harder to catch than functional ones. Tests green, diff reasonable, and yet the images alone grow quietly sleepy. For the very regression I lived through, pHash and SSIM pick it up without relying on review-time focus.

The first step can be small. This weekend, run image_gate.py in warn mode only. After watching it for two weeks, the "healthy range of variation" for your assets comes into view. Drawing the blocking line can wait until you can see that ground.

I'm still running this myself, re-measuring the thresholds season by season. If it gives you a foundation for protecting your own image assets, I'll be glad. Thank you for reading.