A coding agent that reports "done" is making a claim, not handing over proof. The tests it ran might not cover the change. The file it says it edited might be a different copy. The command it says passed might have been piped through something that swallowed the exit code.
Most people who work with these tools end up with some habit for this. Some read the whole diff. Some re-run the tests themselves. Some only trust a result that a second, separate check produced. Others accept "done" for small changes and only dig in when something touches money, data, or anything that is hard to undo.
We are curious where you draw the line:
- What is the one thing you always check before you accept an agent's "done"?
- Is there a kind of change where you have stopped checking, because it was never wrong?
- Have you been burned by a "done" that was not done? What gave it away?
Short answers are welcome. We are collecting the habits that actually catch problems, not the ideal process, so please share yours in the comments below.
Top comments (2)
My list, in order. One: the claim has to arrive with evidence attached — not a summary of the work, the artifact itself or something I can open. Two: I check the diff against the brief, not against the agent's description of the diff; those are different documents more often than I'd like. Three: anything the run says passed, a separate method re-runs — the maker's own test result is a claim, not a check. And the rule underneath all three: the smaller and more confident the 'done', the more I check the unstated parts — what it decided where the brief was silent. The stated work is usually fine. It's the quiet decisions that bill you later.
The one check that caught real failures for me was never the agent's own log. I build Aladdin, a job agent, and the main agent hands each application to a fresh worker. That worker reads the actual page and form itself. It does not see what the first agent claims it did. If the form cannot be read or an answer is unknown, it asks the user instead of guessing. The first agent's summary is never treated as proof. The live page is.