Picture handing the review replies for an app to an agent. Your request is simple: reply to the reviews that need an answer, and leave the rest alone. The drafts that come back are often sorted differently from how you would have done it.
A short one-star review gets a long, careful reply. A feature request gets a stock thank-you. The agent isn't being careless. The sorting you do in your head was never written down, so there was nothing to follow.
Here is what I'd like you to take from this article: write your judgment rules starting from past cases, not from descriptions of your instincts. If you have five finished jobs on hand, you can try this today. No code required.
Why "my usual feel" doesn't transfer
Most of what we do on familiar work never becomes words. When I decide whether to reply to a review, I'm weighing tone, timing, and what the same person wrote before, without noticing I'm doing it.
Telling an agent to "use good judgment" hands that unspoken part over to guesswork. And guesses drift toward the average of the whole world, not the average of my work.
So here's the line I draw now.
What you give an agent is not a personality. It's a set of boundary examples.
"Be careful" and "be polite" decide nothing. "In this case reply, in that case don't" decides a lot.
Step 1: List five recent cases, with outcomes
Pick five real cases where you already made the call. Not hypothetical ones.
Using reviews of a wallpaper app as the example, choose five and write "replied" or "didn't reply" next to each, plus a one-line reason. It's fine if the reason changes while you write. The places where it changes are the valuable part.
| Case | My call | Reason I discovered while writing |
|---|---|---|
| A review describing a specific bug | Reply | I can see how to reproduce it, and I'll want to say when it's fixed |
| A feature request only | Don't reply | A reply reaches no one. Log the request elsewhere |
| An emotional, empty short comment | Don't reply | Replying can make things worse |
| A low rating based on a misunderstanding | Reply | One sentence clearing it up is enough |
| A thank-you review | Don't reply (rarely, do) | Only when they describe how they use the app |
Building this table is where I first put "don't answer emotional, empty comments" into words. The "rarely, do" on the last row was an exception I hadn't known I had.
Step 2: Turn each reason into an "if… then…" sentence
Next, rewrite the reason column so an agent can read it as a condition. The shape is short: if (situation) then (action), except when (exception) then (other action).
Mine ended up looking roughly like this:
When unsure whether to reply, check in this order.
1. A reproducible bug is described -> draft a reply
2. A feature request only -> no reply; add one line to the request list
3. An emotional, empty short comment -> no reply; just report it to me
4. A thank-you -> usually no reply; draft one if it mentions how the app is used
If still unsure, don't draft anything. Add a one-line reason and ask me.That last line matters more than it looks. Deciding in advance what to do when unsure stops the agent from inventing an answer.
Step 3: Replay past cases before you install the rule
This is the step I most want you to keep.
Before registering the text as a rule, pick three older cases you did not use in Step 1. Hand them to the agent as if they were new requests.
All you check is whether its call matches yours. Don't judge the prose.
On a first replay, it's common for one of the three to be off — say, every thank-you review gets a draft. The cause is usually a loose phrase like "mentions how the app is used." Tighten it to "names a feature or an operation at least once" and replay the same three. They should match now.
Fix only the part that drifted. If you rewrite everything, you lose track of which change helped.
Step 4: Decide where exceptions live
Once a rule starts growing, exceptions pile up. If you keep adding them to the main text, it becomes hard to read, and the agent can confuse priorities.
I'd keep the main text to four items and put exceptions in a separate short note, each with a date and one line. For example: "Even for thank-yous, reply if I replied to this person before."
When the note gets long, I either promote an item into the main text or delete it. A monthly look has been enough for me.
Where people usually trip
Three things tend to trip people up at first. I've done all three.
- You write reasons to fit your conclusions. Mix in a case you were unsure about to guard against it
- Your replay cases look too much like the originals. Choose different kinds
- You stop revisiting once it works. Drift shows up after seasons or circumstances change
What to do next
Pick five past cases today and make the table of outcomes and reasons. Turning it into rule text can wait until tomorrow.
I've come to think the range of work I can hand to an agent is about as wide as the range of my own judgment that I can put into words.