When the App Reads an API Response by Position, I Put a Gate in Front of the Upload So Shipped Versions Don't Crash at Launch
A client that reads a server list by index, as a wallpaper app does, leaves old shipped versions in the wild for a long time. Deleting or reordering a field can crash them at launch. This note shows an append-only check and a hash ledger that freezes shipped version directories, with the output from actually running them.
An app that has reached the store cannot be pulled back to suit my schedule. On the day a new version ships, an old one is still running on someone's phone, and it remembers the shape of the server response exactly as it was when it was released.
The wallpaper apps I run as an indie developer read the server's list by position. Item 1 is the ID, item 2 is the title, item 3 is the category. The apps never look at field names. That keeps things light and fast, but the moment the server drops one item or reorders two, a shipped version starts reading a different item than it expects. In my setup, that ends with the old version crashing at launch.
So I narrowed the rule to one sentence: a response that is read by position only ever gets items appended at the tail. No deletions, no reordering. A written rule does not hold by itself, though. It gets broken on days when I let an agent tidy the delivery tree, or late at night by hand. I decided to make a machine hold it.
This note walks through two gates, with the output from running them: a check that compares the shape of a response, and a ledger that freezes shipped version directories.
First, a one-page table of what is being protected
Before any code, I sorted changes into what may change and what may not. "Read by position" breaks in different ways depending on the kind of change.
Server-side change
Effect on shipped versions
Policy
Append rows (new wallpapers) at the end
Old versions just read the new rows
Free to do
Append an item to the end of each row
Old versions never read the unknown position
Allowed (tail only)
Delete an item in the middle
Everything after it shifts one position
Forbidden
Reorder items
The wrong item gets read; easy to miss when types match
Forbidden
Delete or reorder shipped rows
References from favorites and notifications disappear
Forbidden
Update an item whose value naturally moves, such as a view count
The position does not change
Allowed (name the positions explicitly)
Rows three and four are where the accidents live. Adding rows is safe, and adding an item at the tail is safe. The only dangerous thing is moving the meaning of a position that has already shipped. That is the one thing the check has to watch.
Gate one: compare the shipped response with the one you are about to upload
The idea is simple. Keep the response as it was when you shipped, and compare the upload candidate against it. The check looks at four things.
The shortest row has not gotten shorter.
The type at each existing position has not changed.
Treating the first column as the primary key, no shipped row has vanished or moved.
Values on shipped rows are identical, except at positions explicitly marked as allowed to change.
#!/usr/bin/env python3"""出荷済み版の応答(baseline)と、これから上げる応答(candidate)を比べ、「位置で読む」クライアントを壊す変更だけを落とす。使い方: append_only_check.py BASELINE.json CANDIDATE.json [--mutable 3,5]--mutable: 出荷済みの行でも値が変わってよい列の位置(例: 閲覧数)。それ以外は値まで一致を求める終了コード: 0=安全 / 1=壊す変更あり / 2=入力不正"""import json, sysfrom collections import Counterdef kind(v): # 値でなく「型」を見る。壁紙が増えて値が変わるのは正常だから if isinstance(v, bool): return "bool" if isinstance(v, int): return "int" if isinstance(v, float): return "float" if isinstance(v, str): return "str" if v is None: return "null" return type(v).__name__def column_kinds(rows): """列ごとの型の集合(null は「未設定」なので型の判定から外す)""" cols = {} for r in rows: for i, v in enumerate(r): if v is not None: cols.setdefault(i, set()).add(kind(v)) return colsdef main(base_path, cand_path, mutable=frozenset()): try: base = json.load(open(base_path, encoding="utf-8")) cand = json.load(open(cand_path, encoding="utf-8")) assert all(isinstance(r, list) for r in base + cand) except Exception as e: print(f"入力不正: {e}"); return 2 problems = [] # 1) 行ごとの列数: 出荷済みの行より短くしない base_w = Counter(len(r) for r in base) cand_w = Counter(len(r) for r in cand) min_base, min_cand = min(base_w), min(cand_w) if min_cand < min_base: problems.append(f"最短の行が {min_base} 列 → {min_cand} 列に短くなっています") # 2) 既存の列の型が変わっていないか(位置 i の意味が入れ替わっていないか) bk, ck = column_kinds(base), column_kinds(cand) for i in sorted(bk): if i not in ck: problems.append(f"列 {i} が候補から消えています(型 {sorted(bk[i])})") elif not ck[i] <= bk[i]: problems.append(f"列 {i} の型が {sorted(bk[i])} → {sorted(ck[i])} に変わっています") # 3) 先頭列を主キーとみなし、出荷済みの行が消えていないか・並びが入れ替わっていないか base_ids = [r[0] for r in base] cand_ids = [r[0] for r in cand] if cand_ids[:len(base_ids)] != base_ids: gone = [x for x in base_ids if x not in set(cand_ids)] problems.append(f"出荷済みの行の先頭一致が崩れています(消えた行 {len(gone)} 件)" if gone else "出荷済みの行の並びが入れ替わっています") # 4) 同じ型の列どうしの入れ替えは型だけでは見えない。出荷済みの行は値まで一致を見る by_id = {r[0]: r for r in cand} diffs = Counter() for r in base: c = by_id.get(r[0]) if c is None: continue for i, v in enumerate(r): if i in mutable: continue if i >= len(c) or c[i] != v: diffs[i] += 1 for i, n in sorted(diffs.items()): problems.append(f"列 {i}: 出荷済みの行 {n} 件で値が変わっています(入れ替えの疑い)") print(f"baseline: {len(base)} 行 / 列数 {dict(sorted(base_w.items()))}") print(f"candidate: {len(cand)} 行 / 列数 {dict(sorted(cand_w.items()))}") if problems: for p in problems: print("NG " + p) return 1 added_cols = max(cand_w) - max(base_w) print(f"OK 末尾追加のみ(追加行 {len(cand)-len(base)} 件・追加列 {max(added_cols,0)} 本)") return 0if __name__ == "__main__": args = [a for a in sys.argv[1:]] mutable = frozenset() if "--mutable" in args: k = args.index("--mutable") mutable = frozenset(int(x) for x in args[k + 1].split(",") if x) del args[k:k + 2] if len(args) != 2: print(__doc__); sys.exit(2) sys.exit(main(args[0], args[1], mutable))
Here is what I got when I ran it. The baseline is a small 3-row, 4-column response. It is a stand-in to exercise the check, not the real response format.
--- ok (one row added, one column appended to every row)baseline: 3 行 / 列数 {4: 3}candidate: 4 行 / 列数 {5: 4}OK 末尾追加のみ(追加行 1 件・追加列 1 本)exit=0--- drop (the third column removed entirely)baseline: 3 行 / 列数 {4: 3}candidate: 3 行 / 列数 {3: 3}NG 最短の行が 4 列 → 3 列に短くなっていますNG 列 2 の型が ['str'] → ['int'] に変わっていますNG 列 3 が候補から消えています(型 ['int'])NG 列 2: 出荷済みの行 3 件で値が変わっています(入れ替えの疑い)exit=1--- rowgone (the shipped second row removed)NG 出荷済みの行の先頭一致が崩れています(消えた行 1 件)exit=1
(The script's messages are in Japanese, as in my own tooling. They say, in order: the shortest row got shorter; a column's type changed; a column disappeared; values changed on shipped rows; the shipped rows no longer match at the head.)
✦
Thank you for reading this far.
Continue Reading
What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.
WHAT YOU'LL LEARN
✦Before you change a server response, you'll be able to inventory which positions shipped versions read and draw the line between what you may change and what can only be appended at the tail
✦You'll see why comparing types alone misses a swap of same-typed columns, and move a check that compares values on shipped rows into your own pre-upload script
✦By freezing shipped version directories with hashes, you can stop a change that would break old versions before upload, even on a day you let an agent tidy the delivery tree, and skip the time spent chasing crash reports from old versions
Secure payment via Stripe · Cancel anytime
✦
Unlock This Article
Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.
I should be honest here. The first version of this check had no fourth item. It compared types only.
Given a response with the second and third columns swapped, it reported OK. That is one of three broken samples (drop, swap, rowgone) missed, or 33%. Both columns were strings. The types were unchanged, the widths matched, and every row was present. Yet an old version would read the category from the title's position and the title from the category's position. A swap of same-typed columns is invisible to a type comparison by construction.
That was the counterintuitive part. I had assumed that keeping the types meant keeping compatibility. For a client that reads by position, compatibility depends not on the type but on what sits at that position. For shipped rows, the check has to compare the values themselves.
Comparing values then creates a different problem: positions that legitimately move, like a view count, get flagged as changed. So I added --mutable, which names the positions that are allowed to move.
--- view count (4th column) updated, one column and one row appended$ append_only_check.py v100/list.json cand/okmut.json --mutable 3OK 末尾追加のみ(追加行 1 件・追加列 1 本)$ append_only_check.py v100/list.json cand/okmut.json # without --mutableNG 列 3: 出荷済みの行 3 件で値が変わっています(入れ替えの疑い)
--mutable is not an escape hatch. It is a small checkpoint that forces a person to think, at least once, about which positions are allowed to move. If you forget to write it, the check fails, and that failure is the prompt to ask the question.
Three pitfalls in the check itself
In Python, bool is a kind of int.isinstance(True, int) is true, so the type function tests bool first. Reverse the order and a boolean column is treated like an integer column, and a type change slips through.
A JSON 1 and 1.0 count as different types. A column that turns from integer to decimal can crash an old version depending on how it reads the value, and it is easy to miss, so I leaned toward failing it.
Rows with null are excluded from the type comparison. Unset items are common, and failing on each of them would make the gate untrustworthy. The value comparison (the fourth check) catches a change from null to a value.
Any of these, if it produced one false alarm or one miss in production, would cost the gate some trust. As a countermeasure, I turn each case into a deliberately broken sample and confirm the check fails the way I expect before I rely on it.
Gate two: freeze shipped version directories with hashes
Comparing response shapes is not enough on its own. A delivery tree has a directory per version, and shipped versions read files in those directories directly. If I ask an agent to "clean up duplicate files" and it rewrites a shipped version's file along the way, the comparison above never even gets a chance to help.
So on shipping day I record hashes for the whole version directory in a ledger, and from then on I verify mechanically that nothing there has been touched.
#!/usr/bin/env bash# 出荷済みの版ディレクトリを「触らない」ことを、ハッシュの台帳で守る# 使い方: ./freeze_shipped.sh record v100 # 出荷した日に1回# ./freeze_shipped.sh verify # アップロード前に毎回set -euLEDGER="shipped.sha256"case "${1:-}" in record) dir="${2:?版ディレクトリを指定}" # 既に台帳にある版を上書きしない(出荷後に記録し直すと、変更を隠せてしまう) if grep -q " ${dir}/" "$LEDGER" 2>/dev/null; then echo "NG ${dir} は記録済みです。変更が必要なら新しい版を切ってください" >&2; exit 1 fi find "$dir" -type f | LC_ALL=C sort | xargs shasum -a 256 >> "$LEDGER" echo "recorded: $(find "$dir" -type f | wc -l | tr -d ' ') files from ${dir}" ;; verify) # 台帳にある全ファイルを再計算。1つでも違えば非0で終わる shasum -a 256 -c "$LEDGER" --quiet && echo "OK $(wc -l < "$LEDGER" | tr -d ' ') files unchanged" ;; *) echo "usage: $0 record <dir> | verify" >&2; exit 2 ;;esac
If record could be run again after shipping, it could hide a change. So it fails for any version already in the ledger. This is what I got:
$ ./freeze_shipped.sh record v100recorded: 3 files from v100$ ./freeze_shipped.sh record v100 # second timeNG v100 は記録済みです。変更が必要なら新しい版を切ってくださいexit=1$ ./freeze_shipped.sh verifyOK 3 files unchanged$ ./freeze_shipped.sh verify # after rewriting one shipped filev100/list2.json: FAILEDshasum: WARNING: 1 computed checksum did NOT matchexit=1
(The refusal message reads: "v100 is already recorded. If you need to change something, cut a new version.")
When something has to change, cut a new version directory instead of editing the old one. Old versions keep what they read, new versions get the new shape, and each holds its own files. It is an obvious structure, and without a ledger it quietly collapses on a busy day.
Wire both into one pre-upload script
Finally, I tied the two into a single script. If either one fails, it stops there.
If an agent does the upload, put this gate inside the "confirm it passed, then upload" step. When a completion report says "uploaded" but there is no record of the gate passing, I treat the change as not yet verified. The report may well be right; what is missing is any trace of what was checked before it. I wrote about the same habit in Scheduled agent runs: suspect "it never ran" before "it failed".
I measured the weight too: light enough to run before every upload
A gate that runs every time is useless if it is slow, because people stop running it. So I measured it on synthetic data. On Python 3.10 in this environment, I compared a 20,000-row, 7-column baseline (1.25 MB) with a candidate that adds one column to every row plus 100 new rows (1.49 MB). Three runs took 0.29 s, 0.31 s and 0.29 s, using about 29 MB of memory.
For the directory freeze, I created 2,000 JPEG-sized files of 200 KB each (391 MB in total). Building the ledger took 1.83 s and verifying it took 1.82 s. That is not my real file count, but it gives the order of magnitude. If the delivery tree grows, one option is to verify only the last few shipped versions.
What this gate does not protect
So that you don't over-trust it, here is its scope.
A stale baseline means a stale check. The baseline has to be the response of the shipped version. On the day a new version ships, I update the baseline to that version.
It assumes the first column is the primary key. If your response keys on another position, adjust the handling of r[0].
It cannot tell which positions an old version actually reads. That inventory comes from reading the client code by hand. The check only confirms that you have not moved a position that has shipped.
If responses differ per language, keep a baseline per language. One baseline cannot be compared against several languages.
What to take home
The smallest useful routine has three steps.
List the responses that are read by position, and write down the positions that are allowed to move, such as view counts.
On shipping day, keep a sample of the response and a hash of the version directory.
Put an append-only check in front of the upload.
Keeping types intact and keeping the meaning of a position intact are different things. I started out believing the first was enough. I only saw the gap after feeding the check a deliberately broken response. From now on, a new gate gets to fail on a broken sample at least once before I start relying on it.
Share
Thank You for Reading
Antigravity Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.