Purging Cloudflare Didn't Bring Back the New Wallpapers: the Stale JPEGs Were Held by XPageSpeed on the Origin

When only the edge is stale, choose the purge by the count

Let me settle the Cloudflare side first, in order. If you overwrote a handful of images, per-URL purge is enough. The batch where I replaced sources wholesale, though, touched 536 URLs.

Cloudflare's API accepts at most 30 files per purge request. For 536 URLs that is 18 requests. The dashboard's custom purge also offers "prefix," which takes up to 100 entries per request, and one line such as storage.example.com/ios/wallpaper_apps/ukiyo-e/6_5inch/wallpaper — no scheme — covers everything beneath it. If you replace images directory by directory, prefix purge was the faster and more reliable route for me.

What I don't choose is Purge Everything. Every wallpaper falls out of the edge, and by the next morning the whole catalog has gone back to the origin. On a shared-host origin, that is the one kind of return traffic I most want to avoid.

For the per-URL path I keep a small tool that cuts the list into chunks of 30. The staging merge writes the overwritten IDs to cloudflare_purge_urls.txt, and this script reads it.

"""Send cloudflare_purge_urls.txt to Cloudflare purge_cache, 30 files per request.
   The token comes from the environment (never a literal value in an article).
"""
import json
import os
import sys
import time
import urllib.request
 
ZONE_ID = os.environ["CF_ZONE_ID"]
TOKEN = os.environ["CF_API_TOKEN"]          # a token scoped to Zone > Cache Purge only
ENDPOINT = f"https://api.cloudflare.com/client/v4/zones/{ZONE_ID}/purge_cache"
CHUNK = 30                                  # per-request ceiling for file purges
 
 
def purge(files):
    body = json.dumps({"files": files}).encode()
    req = urllib.request.Request(
        ENDPOINT, data=body, method="POST",
        headers={"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"},
    )
    with urllib.request.urlopen(req, timeout=30) as res:
        data = json.load(res)
    if not data.get("success"):
        raise RuntimeError(data.get("errors"))
    return data["result"]["id"]
 
 
def main(path):
    with open(path, encoding="utf-8") as f:
        urls = [line.strip() for line in f if line.strip()]
    # never send the same URL twice; keep the original order
    seen, ordered = set(), []
    for u in urls:
        if u not in seen:
            seen.add(u)
            ordered.append(u)
    total = len(ordered)
    for i in range(0, total, CHUNK):
        chunk = ordered[i:i + CHUNK]
        rid = purge(chunk)
        print(f"[{i + len(chunk):>4}/{total}] purged {len(chunk)} files (request {rid})")
        time.sleep(0.5)                     # stay clear of the rate limit
 
 
if __name__ == "__main__":
    main(sys.argv[1] if len(sys.argv) > 1 else "cloudflare_purge_urls.txt")

Sending the purge is not the end of the job. Afterwards I re-run probe.sh on a few IDs and only close the task once cf= has flipped to MISS or EXPIRED. Whether the purge took effect is confirmed by a response, not by a report — the same line as before.

What the headers showed on the morning the purge didn't help

Back to that September morning. I purged per URL, confirmed cf=MISS, and the image was still old. Looking again at the origin line from probe.sh, the cache-busted response carried etag: W/"PSA-…" and Last-Modified was gone.

PSA- is the weak ETag that PageSpeed-family optimizers attach. On Xserver the feature is called XPageSpeed, and it serves a re-compressed JPEG under the original URL. Even with the new file sitting on disk, if the optimizer is holding an artifact built from the previous version, the origin keeps returning the old picture. When Cloudflare goes back to the origin, that old picture is what it receives.

There was a second discovery here that ran against my intuition. The optimizer's response carried s-maxage=10. I thought I was returning a one-year immutable promise, yet the edge was being told to keep the object for ten seconds. It held on to the old bytes while cutting the edge's effectiveness — for a wallpaper store there is no worse trade. The wallpapers are already encoded at their final quality on delivery; a second compression pass can only degrade them.

The reason s-maxage wins is that Cloudflare, acting as a shared cache, takes s-maxage over max-age when deciding its own edge TTL. The browser received a one-year promise; the edge received a ten-second one — the same header line meant two different things depending on who read it. Since I had never overridden Edge Cache TTL in the dashboard, the origin's word was final. If you do set an override there, check which side wins in your zone before you rely on this reading.

Switching it off in .htaccess: one wrong directive name gives you a 500

There are two ways to deal with it: turn XPageSpeed off in the server panel, or disable it per directory in .htaccess. I switched it off on all five storage directories. There is not a single situation in wallpaper delivery where being optimized helps.

If you go the .htaccess route, the usual mod_pagespeed spelling, ModPagespeed off, returns a 500 on Xserver. The directive name is different. You use Xserver's own XPagespeed off, and wrap it in a check for the control file so the same file survives on a host where the module doesn't exist.

# .htaccess at the delivery root of storage.example.com
# Wallpapers are already at final quality; no re-compression on the origin
<IfFile /var/xpagespeed/xpagespeed_ctl>
  XPagespeed off
</IfFile>
 
# For reference, the promise on static images. An overwrite isn't done until the purge is
<FilesMatch "\.(jpg|jpeg|png|webp)$">
  Header set Cache-Control "public, max-age=31536000, immutable"
</FilesMatch>

Two reasons for the <IfFile> wrapper. One: when a server migration carries the .htaccess to an environment without XPageSpeed, an unknown directive must not produce a 500. Two: the .htaccess lives in a repository, and I want the identical file on all five hosts.

And there is an order. After switching it off, purge Cloudflare once more. Disabling the optimizer alone leaves the edge holding whatever old copy it picked up during one of those ten-second windows. I once switched it off, felt relieved, and saw the old print again the following morning.

Putting the verdict into the script, by name

To avoid a third such morning, I folded the check into a subcommand called checkserver. It samples IDs across a range, runs the three-way comparison from probe.sh, and names each accident by type. A missed category upload — wallpaper_num bumped but categories/*.txt still old — gets counted in the same place.

"""checkserver: compare local / origin (cache-busted) / edge and name each accident.
   --sample sets the sample size, --id-from the first ID.
"""
import argparse
import os
import random
import urllib.request
from dataclasses import dataclass
 
BASE = "https://storage.example.com/ios/wallpaper_apps/ukiyo-e/6_5inch/wallpaper"
LOCAL_DIR = "./_out_ukiyo-e/ios/6_5inch/wallpaper"
 
 
@dataclass
class Probe:
    size: int
    cf_status: str
    age: str
    etag: str
    last_modified: str
 
 
def fetch(url):
    req = urllib.request.Request(url, headers={"User-Agent": "checkserver/1.0"})
    with urllib.request.urlopen(req, timeout=20) as res:
        body = res.read()
        h = {k.lower(): v for k, v in res.headers.items()}
    return Probe(
        size=len(body),
        cf_status=h.get("cf-cache-status", "-"),
        age=h.get("age", "-"),
        etag=h.get("etag", "-"),
        last_modified=h.get("last-modified", "-"),
    )
 
 
def classify(local_size, edge, origin):
    """Order matters: suspect the upstream (local) first."""
    if local_size is None:
        return "MISSING_LOCAL"
    if origin.etag.startswith('W/"PSA-') or origin.last_modified == "-":
        return "ORIGIN_OPTIMIZER"       # stale even when cache-busted: the origin-side optimizer holds it
    if origin.size != local_size:
        return "ORIGIN_STALE"           # missed upload or failed merge
    if edge.size != origin.size and edge.cf_status == "HIT":
        return "EDGE_STALE"             # a purge will fix this
    return "OK"
 
 
def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--sample", type=int, default=20)
    ap.add_argument("--id-from", type=int, default=1)
    ap.add_argument("--id-to", type=int, required=True)
    args = ap.parse_args()
 
    ids = random.sample(range(args.id_from, args.id_to + 1),
                        k=min(args.sample, args.id_to - args.id_from + 1))
    tally = {}
    for wid in sorted(ids):
        local = os.path.join(LOCAL_DIR, f"wallpaper{wid}.jpg")
        local_size = os.path.getsize(local) if os.path.exists(local) else None
        edge = fetch(f"{BASE}/wallpaper{wid}.jpg")
        origin = fetch(f"{BASE}/wallpaper{wid}.jpg?cb={random.randint(1, 10**9)}")
        verdict = classify(local_size, edge, origin)
        tally[verdict] = tally.get(verdict, 0) + 1
        if verdict != "OK":
            print(f"wallpaper{wid}: {verdict}  local={local_size} origin={origin.size} "
                  f"edge={edge.size} cf={edge.cf_status} age={edge.age} etag={origin.etag}")
    print("---")
    for k, v in sorted(tally.items()):
        print(f"{k:<18} {v}")
    if tally.get("ORIGIN_OPTIMIZER"):
        print("→ XPageSpeed is holding the previous version. Switch it off, then purge again")
    if tally.get("EDGE_STALE"):
        print("→ The edge is holding the previous version. Purge by prefix or by URL")
 
 
if __name__ == "__main__":
    main()

The order inside classify is deliberate. Unless you suspect the upstream first, "edge is stale" and "origin is stale" collapse into the same "sizes differ" and you can't tell them apart. The optimizer check goes first because it is the only layer whose staleness survives a cache-busted request.

One run of this check on that September morning surfaced three different accidents at once: a missed category upload, a stale edge, and the optimizer holding old bytes. All three presented as the same symptom — an old picture — so back when I was checking by eye, they looked like one accident.

I now ask the Antigravity agent to run checkserver after every merge and to paste the named lines verbatim into its report. Executing the purge and touching the server panel stay with me. The line about not handing destructive operations to the agent is the one I drew in The Cleanup Step Removed the Working Directory, Not Its Contents, and I use it here unchanged.

An overwrite is one unit of work: purge plus optimizer check

Looking back, the seed of the accident was planted the moment I overwrote an image I had promised would not change for a year, at the same URL. Had I designed for new IDs, most of this article would be unnecessary. If, for the sake of references, you still choose the same URL, the price is more verification, not less.

The line I drew fits in one sentence. Once you overwrite a delivered file, the Cloudflare purge and the XPageSpeed check are part of the same unit of work. Uploading alone finishes nothing.

While I was looking only at Cloudflare, I kept searching inside the edge for why the purge wasn't working. I was searching in the wrong place. Between the edge and my own machine there is a layer the hosting contract inserts without saying so — once I counted it, the mornings with old prints stopped. If you're designing the Cloudflare side from scratch, I'd rather you check what sits inside your origin before you settle the layout in Antigravity × Cloudflare Edge SaaS Architecture Guide.

Start with a single image you serve today. Fetch it twice with curl -sD -, once plain and once with ?cb= appended, and put the etag and Last-Modified lines side by side. If the two responses don't wear the same face, your origin has a layer you weren't counting either.