When CI Passes but App Review Rejects Your Screenshots — Field Notes on Measuring the Freshness of Your Validation Rules

Detect Per-Locale Caption Overflow After Rendering

The second blind spot is caption overflow. A line that fits in Japanese grows by 40 percent in German and escapes the safe area. Generation succeeds, size validation passes, and sometimes review passes too — which is arguably worse than a rejection, because the broken artwork simply goes live on your store page.

If you composite captions yourself, you can measure the rendered text width at composition time.

// scripts/render-captions.js (excerpt)
const { createCanvas, registerFont } = require('canvas');
registerFont('./fonts/NotoSansJP-Bold.otf', { family: 'Caption' });
 
function measureCaption(text, fontSizePx, maxWidthPx) {
  const canvas = createCanvas(10, 10);
  const ctx = canvas.getContext('2d');
  ctx.font = `${fontSizePx}px Caption`;
  const width = ctx.measureText(text).width;
  return { width, overflow: width > maxWidthPx, ratio: width / maxWidthPx };
}
 
const captions = require('../captions.json'); // { locale: { screen01: "text", ... } }
const SAFE_WIDTH = 1320 - 120 * 2; // 6.9-inch width minus margins
 
let failed = false;
for (const [locale, screens] of Object.entries(captions)) {
  for (const [screen, text] of Object.entries(screens)) {
    const m = measureCaption(text, 64, SAFE_WIDTH);
    if (m.overflow) {
      console.error(`❌ ${locale}/${screen}: overflow ${(m.ratio * 100).toFixed(0)}% — "${text}"`);
      failed = true;
    }
  }
}
process.exit(failed ? 1 : 0);

One measured threshold does a lot of work here. In my own asset sets, English captions averaged around 1.2 times the width of the Japanese layout they were designed against, with long lines approaching 1.5 times. So I reserve a safe width of roughly the maximum expected Japanese width times 1.5, and any line that still overflows gets shortened on the translation side. Shrinking the font is the other option, but caption sizes that vary by locale make the store listing look uneven, so I prefer cutting words.

Catch Silent Degradation with Perceptual Diffs Against Golden Images

The third layer catches assets whose dimensions and file sizes are correct but whose contents are broken. The incident that sold me on this: a CI runner image update changed CJK font resolution, and captions silently rendered in a fallback font. Size validation passed, of course. A human would notice in a second; the machine said nothing.

Keep the approved assets in the repository as goldens, and diff every generation against them.

// scripts/perceptual-diff.js
const fs = require('fs');
const { PNG } = require('pngjs');
const pixelmatch = require('pixelmatch');
 
function diffRatio(goldenPath, candidatePath) {
  const golden = PNG.sync.read(fs.readFileSync(goldenPath));
  const cand = PNG.sync.read(fs.readFileSync(candidatePath));
  if (golden.width !== cand.width || golden.height !== cand.height) return 1;
  const diff = new PNG({ width: golden.width, height: golden.height });
  const diffPixels = pixelmatch(
    golden.data, cand.data, diff.data,
    golden.width, golden.height,
    { threshold: 0.12 } // ignore anti-aliasing jitter
  );
  return diffPixels / (golden.width * golden.height);
}
 
const RATIO_LIMIT = 0.02; // more than 2% changed pixels means an unintended change

The 2 percent limit is an empirical line. Recompression and anti-aliasing jitter stay around 0.5 percent; font substitutions and layout breakage show up above 5 percent. The threshold sits between the two. When you change the design on purpose, you let the diff fail, review it, and update the golden — and that golden update, going through code review, becomes your change history for free.

Bind Each Asset Set to a Build Number

The last piece shortens the investigation when a rejection does happen. Emit a manifest with every generation, tying together the commit, the contract version, and the build number that produced the set.

// scripts/write-asset-manifest.js
const manifest = {
  buildNumber: process.env.BUILD_NUMBER,
  commit: process.env.GITHUB_SHA,
  specVersion: require('../store-spec.json').specVersion,
  generatedAt: new Date().toISOString(),
  files: collectSha256('./screenshots/optimized') // per-file hash list
};
fs.writeFileSync('./screenshots/optimized/manifest.json', JSON.stringify(manifest, null, 2));

Rejection emails rarely say more than which screenshots offended, so root-causing depends entirely on your own records. With the manifest, you open the manifest.json of the submitted set: an old specVersion means contract staleness, a recent commit means a layout change. The triage becomes a straight line. Investigations that used to cost me the better part of an hour of scrolling through generation logs now take minutes.

Wiring this into GitHub Actions is unglamorous: freshness check → generation → caption measurement → dimension validation → perceptual diff → manifest, in series. The only deliberate choice is putting the freshness check before generation — if the rules are rotten, everything downstream is theater.

What to Do Next

You don't need all of this at once. The highest-leverage piece is the contract file plus the staleness gate: if you already have a validation script, evict its constants into JSON and add check-spec-freshness.js at the front. Start there, confirm a rejection-free release, then layer in caption measurement and perceptual diffs.

If you've ever lost an evening to a deadline-night rejection, I hope these notes save you the next one.