Imagine this workflow:
- Agent A writes a patch and opens a pull request.
- Agent B reviews it and says "LGTM."
- CI turns green.
Two agents, one approved PR. Sounds like independent review.
Now change a few details.
Both agents can modify the same workspace. The reviewer relies on the author's generated summary. The tested commit might not be the commit that ultimately gets merged.
Do we still have independent verification, or just two agreeing opinions?
I'm increasingly interested in the boundaries that make a review independent, rather than simply the number of models involved.
For example, a reviewer could use a read-only checkout pinned to an exact commit, inspect the raw diff, and verify which tests actually executed. That gives us evidence the implementing agent cannot simply rewrite.
But how much separation is enough?
I'd love to hear how others handle this:
- Is a second AI model with a separate prompt sufficient for low-risk changes?
- Do you require separate workspaces, credentials, and immutable artifacts?
- What evidence makes you trust an AI-generated review?
- Where do you still require human approval?
Share your actual setup, a failure case, or even a simpler approach that works.
I'm especially interested in small, reproducible tests that could challenge our assumptions.
Disclosure: I maintain Agent Role Contracts (ARC), an open-source checker for declared agent responsibilities and task boundaries. Runtime enforcement and independent review require additional controls. This discussion is about what those controls should actually prove.
Top comments (2)
I'd start with a pinned commit for the reviewer. In one feature run my lead agent built a review package from the branch head, then the implementer landed a third commit to finish its fix round, so the package the reviewer was working from was stale. The lead noticed the head had moved and rebuilt it. The next day it started pinning the exact commit before packaging any diff.
Thanks, Jackson! This is exactly the kind of real-world failure case I was hoping to learn from.
Your example highlights an important distinction: pinning the commit before review protects the review input, but we also need to verify that the reviewed commit is still the one being merged.
I'd love to turn this into a small, reproducible test:
Ideally, the review receipt would bind the commit SHA, diff digest, and test evidence together.
That gives us a useful principle: a review can be valid evidence for an earlier commit without being valid approval for the current PR.
One question: have you automated the final HEAD-versus-reviewed-commit check, or is that still handled manually?
I'd be interested in comparing approaches. Your example could make a great reproducible failure-injection case.