Ordering Six App Updates for the API Level 36 Deadline, and Sizing Each Staged Rollout

Auditing All Six at Once

To move that judgment out of my head and into a table, I wrote a script that walks every project, reads the Gradle files, manifests, and version catalogs together, and prints targetSdk plus the surfaces each app touches as one row.

#!/usr/bin/env python3
"""Walk several Android projects and print targetSdk plus touched surfaces as one table.
 
Usage:
    python3 surface_audit.py ~/apps/*/
"""
import re
import sys
from pathlib import Path
 
# Surfaces most exposed to behavior changes: display name -> (detection hint, weight).
SURFACES = {
    "media-write":  (r"WRITE_EXTERNAL_STORAGE|MediaStore\.", 3),
    "fg-service":   (r"FOREGROUND_SERVICE|startForeground", 3),
    "exact-alarm":  (r"SCHEDULE_EXACT_ALARM|setExactAndAllowWhileIdle", 2),
    "notification": (r"POST_NOTIFICATIONS|NotificationManagerCompat", 1),
    "edge-to-edge": (r"enableEdgeToEdge|WindowCompat\.setDecorFitsSystemWindows", 2),
    "native-lib":   (r"externalNativeBuild|\.so\b|ndkVersion", 3),
    "billing":      (r"com\.android\.billingclient", 2),
    "ads":          (r"play-services-ads|applovin|unity-ads|pangle", 2),
}
 
SDK_RE = re.compile(r"(targetSdk(?:Version)?|compileSdk(?:Version)?|minSdk(?:Version)?)"
                    r"\s*(?:=|\s)\s*['\"]?(\d{2})['\"]?")
 
 
def read_text(p: Path) -> str:
    try:
        return p.read_text(encoding="utf-8", errors="ignore")
    except OSError:
        return ""
 
 
def scan(root: Path) -> dict:
    blob = []
    sdk = {}
    for pattern in ("**/build.gradle", "**/build.gradle.kts",
                    "**/AndroidManifest.xml", "**/gradle/libs.versions.toml"):
        for f in root.glob(pattern):
            if "/build/" in str(f):      # skip build output
                continue
            text = read_text(f)
            blob.append(text)
            for key, val in SDK_RE.findall(text):
                sdk.setdefault(key.replace("Version", ""), val)
    joined = "\n".join(blob)
    hits, score = [], 0
    for name, (rx, weight) in SURFACES.items():
        if re.search(rx, joined):
            hits.append(name)
            score += weight
    return {"name": root.name, "sdk": sdk, "hits": hits, "score": score}
 
 
def main() -> None:
    roots = [Path(a).expanduser() for a in sys.argv[1:]]
    if not roots:
        print("Pass at least one project root", file=sys.stderr)
        raise SystemExit(2)
    rows = [scan(r) for r in roots if r.is_dir()]
    rows.sort(key=lambda r: -r["score"])
    print(f"{'app':<22}{'target':>7}{'min':>5}{'risk':>6}  surfaces")
    for r in rows:
        print(f"{r['name']:<22}{r['sdk'].get('targetSdk', '-'):>7}"
              f"{r['sdk'].get('minSdk', '-'):>5}{r['score']:>6}  {','.join(r['hits'])}")
 
 
if __name__ == "__main__":
    main()

The output looks like this:

app                    target  min  risk  surfaces
wallpaper-a                34   24     8  media-write,notification,billing,ads
relax-b                    35   26     5  fg-service,ads
tiny-c                     36   26     0

The weights are not rigorous. I set a surface to 3 if it has personally cost me rework before, and 1 or 2 if the official migration steps were enough. Those numbers are meant to be tuned against your own history.

Two caveats about the script. Projects using a version catalog will not have dependency strings in build.gradle.kts at all, which is why gradle/libs.versions.toml is in the scan list — drop it and an ads SDK silently reports a score of 0. And \.so\b happily matches Gradle comments and unrelated text, so I eyeball the hit list after each run. Trusting an automated audit blindly is how you end up with a confidently wrong ranking.

On my first run one app came back at 0, and the cause was exactly that missed version catalog. Having the pitfall surface on attempt one was lucky.

What matters is not the absolute score. It is that the ranking of the six is now fixed. With the ranking in a table, revisiting it tomorrow produces the same conclusion.

The Ads SDK Changed the Order

The single biggest factor in the audit turned out to be whether an app carries an ads SDK.

Apps with ads get OS changes and ads-SDK updates arriving in roughly the same window. With AdMob mediation across several networks, one adapter bump pulls a chain of dependencies with it. Raise targetSdk in that same release and any crash forces you to ask two questions at once: was it the OS behavior change, or the SDK update?

So I made a rule: never ship an ads SDK bump and a targetSdk bump in the same release. The SDK goes out first and gets a stability window; targetSdk follows in the next release. That costs one extra release, and it halves the number of suspects when something crashes.

I applied the same reasoning to edge-to-edge enforcement and got it out of the way earlier. I wrote up that migration separately in Migrating Wallpaper Apps to Mandatory Edge-to-Edge on targetSdk 36 with Antigravity. Layout breakage is visible to the eye, so it is the kind of change you can judge without waiting for statistics. Separating it out means the deadline release only has crashes to watch.

What a 1% Rollout Actually Tells You

With the order settled, the next question was step size. This is where an assumption of mine fell over.

For years I had shipped at 1%, waited a few days, and widened if nothing looked wrong. I had never checked that arithmetic.

What follows is calculated, not measured. Take a baseline crash rate of 0.5%, define the worsening you refuse to miss as 1.0% (double), and solve for the sessions needed at a one-sided 5% level with 80% power.

Read it in reverse and it gets blunt. With 200 sessions in hand, the smallest worsening you can statistically separate from noise is 0.5% to 4.29%. The build has to break more than eightfold before you can say it was not chance.

Sitting at 1% for a few days, then, is doing almost no statistical work. A staged rollout is not a mechanism for detecting regressions. It is a mechanism for capping the blast radius.

That distinction changes behavior. If you believe it is a detector, "let's watch a bit longer" always sounds reasonable. Once you accept it is a cap, the honest move is to decide how much damage you are willing to accept and move on. I prefer the second framing.

Zero Crashes Is Not the Same as Safe

There was a second thing I had been reading wrong: what a zero-crash result means.

Observe n sessions and see zero events, and the 95% upper bound on the true rate is roughly 3/n — the rule of three.

Zero crashes across 300 sessions only says the rate is under 1%. If the current version sits at 0.5%, a doubling is entirely consistent with what you saw.

"Zero crashes, widening the rollout" is not an argument while n is small. Since working this through, I always report the observation count alongside the crash count. A count on its own reads like good news and carries no information.

Working the Step Size Backwards

Doing this by hand every release does not last, so I put it in a script. Give it a daily session count and it prints, for each rollout percentage, how long a verdict takes and what a zero-crash result would bound the rate to.

#!/usr/bin/env python3
"""Work staged-rollout step size backwards from the observations a decision needs.
 
Usage:
    python3 rollout_sizing.py --dau 12000 --baseline 0.005 --tolerate 2.0
"""
import argparse
from statistics import NormalDist
 
ND = NormalDist()
 
 
def sessions_needed(p0: float, p1: float, alpha: float = 0.05, power: float = 0.80) -> float:
    """Sessions per arm required to detect a move from p0 to p1."""
    if p1 <= p0:
        raise ValueError("p1 must be greater than p0")
    za = ND.inv_cdf(1 - alpha)          # one-sided
    zb = ND.inv_cdf(power)
    pbar = (p0 + p1) / 2
    num = (za * (2 * pbar * (1 - pbar)) ** 0.5
           + zb * (p0 * (1 - p0) + p1 * (1 - p1)) ** 0.5) ** 2
    return num / (p1 - p0) ** 2
 
 
def zero_event_upper_bound(n: int) -> float:
    """95% upper bound on the rate after n observations with zero events."""
    return 3.0 / n if n > 0 else 1.0
 
 
def main() -> None:
    ap = argparse.ArgumentParser()
    ap.add_argument("--dau", type=float, required=True, help="daily sessions for this app")
    ap.add_argument("--baseline", type=float, default=0.005, help="current crash rate")
    ap.add_argument("--tolerate", type=float, default=2.0,
                    help="worsening multiple you refuse to miss")
    ap.add_argument("--max-days", type=float, default=3.0, help="days available per stage")
    args = ap.parse_args()
 
    p0 = args.baseline
    p1 = p0 * args.tolerate
    need = sessions_needed(p0, p1)
 
    print(f"current crash rate   : {p0 * 100:.3f}%")
    print(f"must not miss        : {p1 * 100:.3f}% ({args.tolerate:.1f}x)")
    print(f"sessions needed/arm  : {need:,.0f}")
 
    for pct in (1, 5, 10, 20, 50, 100):
        per_day = args.dau * pct / 100
        days = need / per_day if per_day else float("inf")
        verdict = "ok" if days <= args.max_days else "no"
        bound = zero_event_upper_bound(int(per_day * args.max_days))
        print(f"  {pct:3d}% → {per_day:9,.0f} sessions/day / "
              f"verdict in {days:6.1f} days {verdict} / "
              f"zero over {args.max_days:.0f}d bounds rate at {bound * 100:.2f}%")
 
 
if __name__ == "__main__":
    main()

For an app at 12,000 sessions a day, watching for a doubling:

current crash rate   : 0.500%
must not miss        : 1.000% (2.0x)
sessions needed/arm  : 3,681
    1% →       120 sessions/day / verdict in   30.7 days no / zero over 3d bounds rate at 0.83%
    5% →       600 sessions/day / verdict in    6.1 days no / zero over 3d bounds rate at 0.17%
   10% →     1,200 sessions/day / verdict in    3.1 days no / zero over 3d bounds rate at 0.08%
   20% →     2,400 sessions/day / verdict in    1.5 days ok / zero over 3d bounds rate at 0.04%
   50% →     6,000 sessions/day / verdict in    0.6 days ok / zero over 3d bounds rate at 0.02%
  100% →    12,000 sessions/day / verdict in    0.3 days ok / zero over 3d bounds rate at 0.01%

With two weeks until the deadline, 1% and 5% drop out immediately: the verdict arrives after the cutoff. I moved to 10% for three days, then 50%.

Smaller apps come out worse. At 900 sessions a day, even a 100% rollout needs 5.1 days to detect a doubling. In that case the right move is to stop pretending statistics will decide it and read every crash by hand instead. Low volume is exactly what makes reading all of them possible.

Write the Rollback Conditions Before You Publish

Once the step size is fixed, the next decision falls out of it: what stops the rollout.

I now write these three lines into a note before pressing publish on each release.

1. Halt: a single new crash signature, at any count, blocks the next stage
2. Roll back: crash rate above 2x the current version and more than 1,000 sessions observed means reverting the same day
3. Proceed: neither of the above, and observations have reached the required session count

For a smaller release you can lower that 1,000-session figure in rule two. If you do, I recommend checking what the reduced number actually supports via the rule of three before adopting it. Loosening a threshold is fine; loosening it without noticing is not.

Deciding this after publishing always bends optimistic. "Probably device-specific." "It might settle down." That pull exists because the person judging the release is also the person who built it. Writing it down first takes most of the wobble out.

What I Delegated, and What I Kept

Of all this work, what I handed to an Antigravity agent was investigation and evidence gathering.

Delegated:

• Running the cross-project audit and formatting the results into a table
• Listing call sites for the APIs likely to be affected, with file names and line numbers
• Collecting the relevant sections of official documentation per change
• Drafting release notes

Kept:

• Choosing the rollout percentage
• Pressing publish
• Deciding whether to roll back

The dividing line is whether a mistake is cheap to catch. An audit result that is wrong announces itself: open a listed file that does not exist and you know in a second; a line number that has drifted takes a few more. A wrong rollout percentage, by contrast, only reveals itself through damage.

CLI 1.1.13 also opened manage_task to declarative custom agents, so background tasks can be listed and stopped from them. That makes it more comfortable to fire off a longer audit and work on something else while it runs. My rule is to only delegate work I know I can stop.

Start With the Fragile One

Back to that first instinct. "Start with the app that has the most users" was exactly wrong.

Update the fragile one first and whatever you find there transfers directly to the remaining five. Update the popular one first and you get maximum exposure with minimum learning. Putting the learning early and the exposure late buys real slack out of the same deadline.

Run surface_audit.py across your projects in one go. The moment the ranking exists as a table, the order you were agonizing over decides itself.