An eighth from my side, earned the slow way: the code handles exactly the case you described, and quietly decides the case you left unstated. No error, no TODO, just a reasonable-looking choice buried in the middle — a default, a sort order, an empty-input behaviour — that you only discover weeks later when it disagrees with what you actually wanted. It's the hardest almost-right to catch because review checks what you asked for. The unstated decisions are the ones nobody reviews, including the person who wrote the prompt. My fix is dull: before a run, I write down my assumptions where the agent can contradict them, and after, I diff the result against that list.
Software Engineer · Next.js · React · TypeScript · Three.js · Node.js · Spring Boot · MySQL · WordPress · Shopify · Claude Code · Codex · Building tools to make your daily dev work easier
Thank you, this is a sneaky one, because the code is right for everything you actually said. One cheap way to surface those hidden choices is to ask the agent at the end to list every decision it made that the prompt didn't specify: defaults, ordering, what happens on empty input. It won't catch everything, but it turns a buried choice into a line you can disagree with. Do you keep your assumptions in the prompt itself, or in a separate file the agent reads?
Separate file, mostly. The standing assumptions — the ones that are true for every run — live in a rules file the agent reads at the start, so I don't retype them into every prompt and they don't drift between runs. The assumptions specific to one task go in that task's brief, in a short list the agent is told to contradict if they're wrong. And your end-of-run trick is the third piece: the decisions the agent made that nobody specified get written into the run record. So three layers: standing file, per-task list, and the run's own log catching what both missed. The prompt itself is the worst place for assumptions, in my experience — it's the one text nobody re-reads.
Software Engineer · Next.js · React · TypeScript · Three.js · Node.js · Spring Boot · MySQL · WordPress · Shopify · Claude Code · Codex · Building tools to make your daily dev work easier
Thank you, that three-layer setup makes a lot of sense. A rule in a file gets looked at again every time the file changes, while the same rule buried in a prompt is gone after one run. And using the run log to catch what both lists missed closes the loop nicely.
Glad it landed. One small addition from the scar tissue: the run-log layer only works if it's read on a schedule, not when something already looks wrong. I do a one-minute pass over the logs on weekday mornings and save the real judgement for the weekend. The logs that get read late are just archaeology. The ones read daily actually change the next run.
Software Engineer · Next.js · React · TypeScript · Three.js · Node.js · Spring Boot · MySQL · WordPress · Shopify · Claude Code · Codex · Building tools to make your daily dev work easier
"Logs read late are just archaeology" is going on a sticky note. I'm taking the one-minute weekday pass too. Do you read the raw logs, or does the agent write you a short summary at the end of each run?
The “right shape, wrong value” one stuck with me most — a filled-in field with no exception is exactly the kind of failure that slips past a green test run. Maxwell’s check of comparing five random rows with the source by hand, labels and units included, is wonderfully cheap. Have you found any of these seven checks lend themselves to running automatically in CI, or do they mostly stay manual 30-second habits?
Software Engineer · Next.js · React · TypeScript · Three.js · Node.js · Spring Boot · MySQL · WordPress · Shopify · Claude Code · Codex · Building tools to make your daily dev work easier
Thank you! Most of them can move into CI, and the one that can't is the interesting part. Artur's heavy package is the easiest: a bundle size limit fails the build when a new dependency adds too much. Guillermo's new folder that never ships becomes a check of the built output against a list of files that must be there, like Mateo suggested above. Khali's stale closure is mostly caught by the exhaustive-deps lint rule. A raised timeout can be a tiny diff check that flags any timeout value that went up.
Maxwell's right shape, wrong value is the one I'd keep human. A test can compare against a saved sample, but once someone picks the sample rows they stop being random. So the five-row check stays a habit and CI does the rest. Which one would you automate first?
I'd start with Artur's heavy package too — a bundle size limit is the most mechanical of the lot, it fails the build with a clear number attached, and it catches the problem at the exact moment someone adds the dependency. The raised-timeout diff check would be my second pick for the same reason: it's a tiny, objective signal. The five-row check staying human makes sense to me; the moment the sample rows are chosen by hand, the randomness that makes the check work is gone.
Software Engineer · Next.js · React · TypeScript · Three.js · Node.js · Spring Boot · MySQL · WordPress · Shopify · Claude Code · Codex · Building tools to make your daily dev work easier
Glad we landed on the same two. One small thing I learned with size limits: set the limit just above today's size, not with a big margin. Then it fails on the one commit that adds the heavy package, instead of quietly letting three medium ones through first.
Building AI systems and fixing AI-generated apps before they collapse in production.
Working with agentic AI, RAG pipelines, AI orchestration, and startup MVP scaling at IT Path Solutions.
The “passes everything, and never ships” example is particularly important because it exposes a gap that code-level validation often misses. A change can be logically correct, have passing tests, and still be absent from the actual deployment artifact.
At IT Path Solutions, this is one of the areas we pay attention to when stabilizing AI-generated applications. I'd add one more CI check: validate the final build or container against the expected artifact manifest, including newly added files and required configuration. Also, test changes deserve independent scrutiny otherwise, an agent can unintentionally make the implementation and its verification agree on the same wrong assumption. The real definition of done should be that the intended behavior survives the entire path to production.
Software Engineer · Next.js · React · TypeScript · Three.js · Node.js · Spring Boot · MySQL · WordPress · Shopify · Claude Code · Codex · Building tools to make your daily dev work easier
Thank you, Mateo. Checking the final artifact against a manifest is a nice way to make Guillermo's check automatic: once the expected file list lives in the repo, a missing folder fails CI instead of waiting for someone to read the deploy script. The one thing I'd watch is keeping that manifest up to date, or it turns into one more list nobody remembers to edit. Do you generate it from the build itself, or maintain it by hand?
We're a place where coders share, stay up-to-date and grow their careers.
An eighth from my side, earned the slow way: the code handles exactly the case you described, and quietly decides the case you left unstated. No error, no TODO, just a reasonable-looking choice buried in the middle — a default, a sort order, an empty-input behaviour — that you only discover weeks later when it disagrees with what you actually wanted. It's the hardest almost-right to catch because review checks what you asked for. The unstated decisions are the ones nobody reviews, including the person who wrote the prompt. My fix is dull: before a run, I write down my assumptions where the agent can contradict them, and after, I diff the result against that list.
Thank you, this is a sneaky one, because the code is right for everything you actually said. One cheap way to surface those hidden choices is to ask the agent at the end to list every decision it made that the prompt didn't specify: defaults, ordering, what happens on empty input. It won't catch everything, but it turns a buried choice into a line you can disagree with. Do you keep your assumptions in the prompt itself, or in a separate file the agent reads?
Separate file, mostly. The standing assumptions — the ones that are true for every run — live in a rules file the agent reads at the start, so I don't retype them into every prompt and they don't drift between runs. The assumptions specific to one task go in that task's brief, in a short list the agent is told to contradict if they're wrong. And your end-of-run trick is the third piece: the decisions the agent made that nobody specified get written into the run record. So three layers: standing file, per-task list, and the run's own log catching what both missed. The prompt itself is the worst place for assumptions, in my experience — it's the one text nobody re-reads.
Thank you, that three-layer setup makes a lot of sense. A rule in a file gets looked at again every time the file changes, while the same rule buried in a prompt is gone after one run. And using the run log to catch what both lists missed closes the loop nicely.
Glad it landed. One small addition from the scar tissue: the run-log layer only works if it's read on a schedule, not when something already looks wrong. I do a one-minute pass over the logs on weekday mornings and save the real judgement for the weekend. The logs that get read late are just archaeology. The ones read daily actually change the next run.
"Logs read late are just archaeology" is going on a sticky note. I'm taking the one-minute weekday pass too. Do you read the raw logs, or does the agent write you a short summary at the end of each run?
The “right shape, wrong value” one stuck with me most — a filled-in field with no exception is exactly the kind of failure that slips past a green test run. Maxwell’s check of comparing five random rows with the source by hand, labels and units included, is wonderfully cheap. Have you found any of these seven checks lend themselves to running automatically in CI, or do they mostly stay manual 30-second habits?
Thank you! Most of them can move into CI, and the one that can't is the interesting part. Artur's heavy package is the easiest: a bundle size limit fails the build when a new dependency adds too much. Guillermo's new folder that never ships becomes a check of the built output against a list of files that must be there, like Mateo suggested above. Khali's stale closure is mostly caught by the exhaustive-deps lint rule. A raised timeout can be a tiny diff check that flags any timeout value that went up.
Maxwell's right shape, wrong value is the one I'd keep human. A test can compare against a saved sample, but once someone picks the sample rows they stop being random. So the five-row check stays a habit and CI does the rest. Which one would you automate first?
I'd start with Artur's heavy package too — a bundle size limit is the most mechanical of the lot, it fails the build with a clear number attached, and it catches the problem at the exact moment someone adds the dependency. The raised-timeout diff check would be my second pick for the same reason: it's a tiny, objective signal. The five-row check staying human makes sense to me; the moment the sample rows are chosen by hand, the randomness that makes the check work is gone.
Glad we landed on the same two. One small thing I learned with size limits: set the limit just above today's size, not with a big margin. Then it fails on the one commit that adds the heavy package, instead of quietly letting three medium ones through first.
The “passes everything, and never ships” example is particularly important because it exposes a gap that code-level validation often misses. A change can be logically correct, have passing tests, and still be absent from the actual deployment artifact.
At IT Path Solutions, this is one of the areas we pay attention to when stabilizing AI-generated applications. I'd add one more CI check: validate the final build or container against the expected artifact manifest, including newly added files and required configuration. Also, test changes deserve independent scrutiny otherwise, an agent can unintentionally make the implementation and its verification agree on the same wrong assumption. The real definition of done should be that the intended behavior survives the entire path to production.
Thank you, Mateo. Checking the final artifact against a manifest is a nice way to make Guillermo's check automatic: once the expected file list lives in the repo, a missing folder fails CI instead of waiting for someone to read the deploy script. The one thing I'd watch is keeping that manifest up to date, or it turns into one more list nobody remembers to edit. Do you generate it from the build itself, or maintain it by hand?