The night my agent shipped nothing: giving generation agents an abstain outcome
When you score a background generation agent by how much it produces, the quality gate quietly loosens over time. Here is a three-valued ACCEPT / ABSTAIN / REJECT design that counts a zero-artifact run as a success, with the code and the measurements from running it.
A little past 2 a.m., the completion notice for a scheduled run arrived. The payload: "artifacts produced: 0." Back then I treated that as a failure, and my first habit each morning was to start digging into why.
But those zero artifacts were not an error. Candidates had been generated, and the quality gate had rejected every one of them. The gate had done exactly its job. My operations were simply wired around the assumption that zero equals broken.
This article is the record of inverting that assumption. For a background generation agent — the kind you run as an Antigravity 2.0 scheduled task or managed agent, autonomously producing artifacts like posts, assets, or reports — what breaks when you make "how much it produced" the success metric? And how does operations change once the gate can return a third outcome: abstain? I will walk through it with the code I actually run.
Score by throughput, and the gate always loosens
When you first wire up a generation agent, the metric you reach for is artifacts per run. It is easy to chart, and the up-and-down is legible at a glance.
That is where the trap is. The moment production count becomes the success signal, a zero-artifact run becomes a failure. Failures pile up, alerts fire, and the operator quietly reacts. There are two available moves: improve the inputs (topics, context) or loosen the gate. The first takes time; the second is a one-line change.
Under a deadline, tired, people pick the cheap move. Nudge the threshold down. Wave one candidate through "just this once." And so the gate loosens, without anyone intending harm, a little at a time.
The generation pipeline I run as an indie developer, for my own apps, hit exactly this. During the stretch when I chased production count, something shipped almost every night — and some of what shipped passed the syntactic checks while leaving the reader with nothing. Count went up, value thinned out. The metric looked healthy while the system was wasting away.
The fix was to change the design of the metric itself. The target is "net positive, or nothing." Ship only artifacts that provably raise the system's average quality; otherwise ship nothing. A mediocre single artifact is a negative, and zero is zero. Zero ranks above negative — that ordering has to be written into the code.
Make the gate three-valued: ACCEPT / ABSTAIN / REJECT
The starting move is to change the gate's return from binary (pass/fail) to three-valued.
Verdict
Meaning
Contribution to the run
ACCEPT
Judged net positive. Safe to publish
Commits one artifact
ABSTAIN
Not bad, but not clearly net positive. Skip it this time
Zero artifacts, but not a failure
REJECT
Requirement violation, broken, or duplicate. Must not pass
Zero artifacts; target for regeneration
A binary world has no ABSTAIN. So the most common state — "not bad, but not worth shipping" — gets shoved onto the ACCEPT side. The point of going three-valued is to give that middle band a home of its own.
from __future__ import annotationsfrom dataclasses import dataclass, fieldfrom enum import Enumclass Verdict(str, Enum): ACCEPT = "accept" ABSTAIN = "abstain" REJECT = "reject"@dataclassclass GateResult: verdict: Verdict reasons: list[str] = field(default_factory=list) signals: dict[str, float] = field(default_factory=dict)class AcceptanceGate: """Score a candidate three ways. REJECT = violation, ABSTAIN = below net-positive.""" def __init__(self, *, min_utility_signals: int = 3, novelty_floor: float = 0.35): self.min_utility_signals = min_utility_signals self.novelty_floor = novelty_floor def evaluate(self, candidate: "Candidate") -> GateResult: reasons: list[str] = [] # --- 1) Hard requirements: any one missing -> REJECT (before net-positive) --- if candidate.has_broken_frontmatter(): reasons.append("broken frontmatter (YAML parses as a map, 500s)") if candidate.duplicates_existing(threshold=0.90): reasons.append("near-identical to an existing artifact (dilution, not gain)") if candidate.violates_policy(): reasons.append("policy violation (banned terms, out-of-scope drift)") if reasons: return GateResult(Verdict.REJECT, reasons) # --- 2) Net-positive test: if unmet, ABSTAIN (skip, do not fail) --- utility = candidate.count_utility_signals() # runnable code / measurements / steps... novelty = candidate.novelty_score() # delta against the existing corpus signals = {"utility": float(utility), "novelty": novelty} if utility < self.min_utility_signals: reasons.append(f"utility signals short {utility}/{self.min_utility_signals}") if novelty < self.novelty_floor: reasons.append(f"novelty below floor {novelty:.2f} < {self.novelty_floor}") if reasons: return GateResult(Verdict.ABSTAIN, reasons, signals) return GateResult(Verdict.ACCEPT, ["judged net positive"], signals)
There is one thing I deliberately do not do here: auto-promote an ABSTAIN candidate to ACCEPT because "it's so close." Auto-promoting near-misses only invites the looseness back inside the code. Let the middle band be the middle band and skip it honestly. That restraint pays off later.
✦
Thank you for reading this far.
Continue Reading
What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.
WHAT YOU'LL LEARN
✦A three-valued gate that returns ACCEPT / ABSTAIN / REJECT with reasons, and deliberately refuses to auto-promote near-misses
✦A run classifier that decouples success from artifact count, plus abstention rate read as a leading indicator
✦A fail-closed grounding check that abstains when reference data is missing or stale, so unattended nights fail loud instead of silent
Secure payment via Stripe · Cancel anytime
✦
Unlock This Article
Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.
Even with a three-valued gate, if you judge the whole run by "was there at least one ACCEPT," you have slid right back to a throughput metric. Define the run outcome on a different axis from production count.
from collections import Counterclass RunOutcome(str, Enum): PRODUCED = "produced" # >= 1 ACCEPT. net positive happened HEALTHY_ABSTAIN = "healthy_abstain" # all ABSTAIN. a correct, successful skip DEGRADED = "degraded" # REJECT-dominated. input or model anomaly BLOCKED = "blocked" # never entered generation (see below)def classify_run(results: list[GateResult]) -> RunOutcome: if not results: return RunOutcome.BLOCKED tally = Counter(r.verdict for r in results) accepts = tally[Verdict.ACCEPT] rejects = tally[Verdict.REJECT] if accepts >= 1: return RunOutcome.PRODUCED # zero ACCEPT still means different things by composition if rejects > len(results) // 2: # more than half were violations = not the gate's win, an input anomaly return RunOutcome.DEGRADED # mostly ABSTAIN = a healthy conclusion that nothing was worth shipping return RunOutcome.HEALTHY_ABSTAIN
The crux is treating HEALTHY_ABSTAIN as a success state. The only outcomes that page you are DEGRADED and BLOCKED. A night that ends with every candidate abstained just leaves a quiet log line and goes back to sleep. That 2 a.m. notification stopped firing because of this single classification.
There is a reason DEGRADED is kept separate too. The surface looks the same — zero ACCEPT — but "skipped" and "everything came back broken" call for entirely different moves. The former means change the inputs. The latter means suspect the model, the prompt, or corrupted reference data. If you do not decompose the surface number, you will confuse the two.
Read abstention rate as a leading indicator, not a failure
Once run outcomes are separated, the next axis is time. The single number I lean on most is not production count but the abstention rate — ABSTAIN over total candidates — and its trend.
@dataclassclass AbstentionWindow: """Track abstention rate over the last N runs and warn on trend.""" window: int = 14 history: list[float] = field(default_factory=list) def record(self, results: list[GateResult]) -> None: if not results: return abstains = sum(1 for r in results if r.verdict is Verdict.ABSTAIN) self.history.append(abstains / len(results)) self.history = self.history[-self.window:] def rate(self) -> float: return sum(self.history) / len(self.history) if self.history else 0.0 def trend_signal(self) -> str: if len(self.history) < self.window: return "collecting" recent = sum(self.history[-3:]) / 3 base = sum(self.history[:-3]) / (len(self.history) - 3) if recent > base + 0.20: # a rising abstention rate is usually topic saturation, not quality loss return "input_saturation" # -> change inputs, leave the gate alone if recent < base - 0.20 and self.rate() < 0.10: return "gate_too_loose" # -> too much is passing, revisit the gate return "stable"
When trend_signal returns input_saturation, intuition whispers "the gate is too strict, loosen it." In practice the opposite held. A rising abstention rate is almost always a sign that you have written the topic area dry, and the right response is to swap the inputs — the topic source, the grounding data. Loosen the gate here and you merely stack thin artifacts onto a saturated field.
Guaranteeing grounding freshness before a run pairs well with the idea in measuring generation opportunity with a freshness SLO. Abstention rate reports input saturation; the freshness SLO reports input decay — two different angles on the same input.
When grounding is missing, abstain fail-closed
The scariest thing in unattended execution is generation running while the reference data is missing. Handed an empty file or a truncated context, an agent does not fall silent and stop — it fills in something plausible and ships it. At night, that piles up until morning.
Make this fail-closed. If the inputs are not satisfied, do not even enter generation; end the run as BLOCKED. Not even ABSTAIN — stop short of the gate.
@dataclassclass GroundingCheck: required_paths: list[str] min_bytes: int = 512 max_age_hours: float = 48.0 def ready(self, fs, now_ts: float) -> tuple[bool, list[str]]: problems: list[str] = [] for path in self.required_paths: meta = fs.stat(path) if meta is None: problems.append(f"missing: {path}") # never fail silently elif meta.size < self.min_bytes: problems.append(f"no substance ({meta.size}B): {path}") elif (now_ts - meta.mtime) / 3600 > self.max_age_hours: age = (now_ts - meta.mtime) / 3600 problems.append(f"stale ({age:.0f}h): {path}") return (len(problems) == 0, problems)def run_generation(candidates_fn, gate, grounding, fs, now_ts, log): ok, problems = grounding.ready(fs, now_ts) if not ok: # do not enter generation on inputs that cannot guarantee net positive log.warn("BLOCKED: grounding not ready", extra={"problems": problems}) return RunOutcome.BLOCKED, [] results = [gate.evaluate(c) for c in candidates_fn()] outcome = classify_run(results) log.info("run complete", extra={"outcome": outcome, "n": len(results)}) return outcome, results
max_age_hours is there because it is not only absence that is dangerous — a reference file whose updates have stalled is dangerous too. "Present, but three days old" is worse than missing precisely because it is harder to notice. Folding freshness into the fail-closed decision keeps quiet staleness from eroding quality.
Three things that ran counter to my expectations
After a few weeks on the three-valued gate and the net-positive SLO, three things came out the opposite of what I had predicted.
One. Raising production count made outcomes worse. During the period I pushed toward shipping at least one artifact every night, I was mixing should-abstain candidates into ACCEPT, and the average quality drifted down. From the day I allowed zero, the density of what passed went up and the downstream metrics turned toward recovery. Give yourself the freedom not to produce, and what you do produce gets better. You know this in your head, and yet your hand still reaches for the button every time the dashboard shows a zero.
Two. A rising abstention rate was usually good news. I braced for it as a sign of decline, but in practice it almost always meant "you've written that topic enough." The move to make is swapping inputs, not relaxing the gate. Had I read abstention as a failure rate, I would have trampled that signal every single time.
Three. Auto-repairing near-misses bred silent corruption. I once added a step that mechanically edited ABSTAIN candidates up to ACCEPT, and pulled it out fast. Repair only erased the surface violation marker; the underlying "not net positive" state remained, and the traces of it were hard to find later. Surfacing the violation at the gate and regenerating — rewritten in its own specific context — turned out faster and cleaner. That call is of a piece with treating candidates in a three-pattern validation loop.
The order to adopt this, and the first line to hold
You do not need to swap everything in at once. The lightest start is to drop in classify_run alone and take HEALTHY_ABSTAIN off the alert list. That by itself stops the nightly false alarms.
Next, add abstention-rate recording, watch two weeks of it, and only then enable trend detection. Set thresholds before you have a feel for the numbers and you will usually miss. Last, add the grounding fail-closed. One silent-failure incident and its value sinks in for good.
There is only one line to hold at the start: do not put "how much it produced" at the top of your success metrics. Hold that and the gate is freed from the pressure to loosen. You become able to count a zero-artifact night, quietly, as a success. Running an agent solo, that quiet on its own lowers the cost of operations.
I am still tuning this myself, but I hope it helps the design of anyone facing an unattended generation pipeline of their own. Thank you for reading.
Share
Thank You for Reading
Antigravity Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.