A pull request passes independent review. The reviewer signs a PASS receipt. The implementation branch advances by one commit, but the old receipt is still available.
What, exactly, does that signature approve?
We build Agent Role Contracts (ARC) at Netsujo. ARC validates the consistency of declared roles, scopes, review relationships, and handoffs. Its declaration-level PASS does not authenticate reviewer keys or authorize a merge. Those are responsibilities of the surrounding runtime and integration gate.
This second Failure Lab is a host-gate companion experiment, not a claim that ARC implements cryptographic review receipts. It is small enough to run with Node.js alone.
One receipt, five decisions
Our example gives an authorized reviewer a temporary Ed25519 key pair and pins its public key in a tiny verifier. The receipt includes a run identity, exact candidate SHA, policy ID, and verdict. Its signature covers all four fields using an explicitly ordered serialization.
We test the following behavior:
| Case | Change | Expected decision |
|---|---|---|
| Matching reviewer receipt | Candidate A, policy v1, trusted key | EVIDENCE_MATCHED_ONLY |
| Stale candidate | Present A's valid receipt while checking candidate B | DENY_STALE_CANDIDATE |
| Changed policy | Present v1's valid receipt under policy v2 | DENY_STALE_POLICY |
| Different signer | Implementer signs the same fields with another key | DENY_UNTRUSTED_SIGNER |
| Missing receipt | No verifiable input | PENDING_MISSING_RECEIPT |
The success condition is intentionally called EVIDENCE_MATCHED_ONLY. A valid receipt is evidence for a particular subject. It is not by itself permission to merge, deploy, or perform external actions.
Run the fixture
Save the following as review-receipt-fixture.mjs and run it with Node.js 22 or later:
node review-receipt-fixture.mjs
import assert from 'node:assert/strict';
import { generateKeyPairSync, sign, verify } from 'node:crypto';
// Synthetic identifiers; no real PR, credentials, agent or network request.
const shaA = 'a'.repeat(40);
const shaB = 'b'.repeat(40);
const runId = 'lab-run-001';
const policyV1 = 'review-policy-v1';
const policyV2 = 'review-policy-v2';
const reviewer = generateKeyPairSync('ed25519');
const implementer = generateKeyPairSync('ed25519');
// Deliberately fixed serialization for this small educational fixture.
// A production protocol needs a specified canonical encoding and key lifecycle.
function signedBytes(r) {
return Buffer.from(JSON.stringify([
1, r.runId, r.candidateSha, r.policyId, r.verdict
]), 'utf8');
}
function issue(receipt, privateKey) {
return {
receipt,
signature: sign(null, signedBytes(receipt), privateKey),
};
}
function inspect({ signedReceipt, currentSha, currentPolicy, pinnedReviewerKey }) {
if (!signedReceipt?.receipt || !Buffer.isBuffer(signedReceipt.signature)) {
return 'PENDING_MISSING_RECEIPT';
}
const { receipt, signature } = signedReceipt;
if (!verify(null, signedBytes(receipt), pinnedReviewerKey, signature)) {
return 'DENY_UNTRUSTED_SIGNER';
}
if (receipt.candidateSha !== currentSha) return 'DENY_STALE_CANDIDATE';
if (receipt.policyId !== currentPolicy) return 'DENY_STALE_POLICY';
if (receipt.verdict !== 'PASS') return 'DENY_REVIEW_NOT_PASS';
return 'EVIDENCE_MATCHED_ONLY';
}
const receipt = {
runId,
candidateSha: shaA,
policyId: policyV1,
verdict: 'PASS',
};
const signedByReviewer = issue(receipt, reviewer.privateKey);
const signedByImplementer = issue(receipt, implementer.privateKey);
const cases = [
['matching reviewer receipt', signedByReviewer, shaA, policyV1, 'EVIDENCE_MATCHED_ONLY'],
['valid receipt replayed on newer commit', signedByReviewer, shaB, policyV1, 'DENY_STALE_CANDIDATE'],
['valid receipt after policy change', signedByReviewer, shaA, policyV2, 'DENY_STALE_POLICY'],
['implementer uses its own signing key', signedByImplementer, shaA, policyV1, 'DENY_UNTRUSTED_SIGNER'],
['receipt missing', null, shaA, policyV1, 'PENDING_MISSING_RECEIPT'],
];
for (const [label, input, currentSha, currentPolicy, expected] of cases) {
const actual = inspect({
signedReceipt: input,
currentSha,
currentPolicy,
pinnedReviewerKey: reviewer.publicKey,
});
assert.equal(actual, expected, label);
console.log(`PASS: ${label} -> ${actual}`);
}
console.log('5/5 fixture assertions passed. No merge or runtime execution was authorized.');
We executed this fixture on November 17, 2026, using Node.js v22.16.0. The five local assertions passed:
PASS: matching reviewer receipt -> EVIDENCE_MATCHED_ONLY
PASS: valid receipt replayed on newer commit -> DENY_STALE_CANDIDATE
PASS: valid receipt after policy change -> DENY_STALE_POLICY
PASS: implementer uses its own signing key -> DENY_UNTRUSTED_SIGNER
PASS: receipt missing -> PENDING_MISSING_RECEIPT
5/5 fixture assertions passed. No merge or runtime execution was authorized.
The negative cases are successful tests: the simulated gate rejects exactly what it should reject.
Why the key boundary matters
The important control here is not the claimedProducer text inside a JSON receipt. The verifier compares the signature against a pinned reviewer public key. A receipt signed with the implementer's separate key does not satisfy that trust condition.
In this lab both key pairs are created by one local process. That makes the exercise reproducible, but it does not demonstrate operational independence. A production system would need to establish who can use the reviewer's private key, how the key is registered or revoked, and whether the review actually occurred. Signature verification alone cannot prove those facts.
The serialized byte format in this toy is also fixed only for the demonstration. A real protocol must specify canonical encoding, key IDs, algorithm choices, policy transitions, and how the verifying authority obtains the trusted inputs.
Where an actual merge gate goes further
To rely on a review receipt, an integration gate should establish at least:
- Subject: the receipt covers the exact candidate revision and relevant base or merge result.
- Authority: the reviewer's identity and key were authorized for that task when the receipt was issued and remain acceptable under the current policy.
- Coverage: all required review and CI obligations are satisfied; missing evidence remains missing.
- Currentness: a new commit, policy revision, or invalidated assumption triggers the applicable re-verification rather than inheriting an old PASS.
- Acceptance: the protected merge and any production verification remain their own gates.
A reused signature is not inherently malicious. Reusing it for a different subject or authority state is the problem. The host must detect that difference.
One more failure to propose
Our five cases do not yet test concurrent admission of the same receipt or a lost response after a side-effecting action. Those require durable identity, deduplication, and external-effect reconciliation beyond a local signature check. We would treat them as separate follow-on tests.
What is the smallest fixture that would falsify your own review gate? A stale candidate, a forged producer, a changed policy, or something else?
We welcome public, synthetic counterexamples in ARC's GitHub repository. Please omit credentials, customer data, and private system paths.
Scope disclosure: the results above come from a self-contained local fixture authored by Netsujo. They are not third-party validation, a test of live reviewer key custody, or proof of ARC runtime enforcement.
Top comments (4)
The forged producer is the fixture that falsifies my gate. A verdict there is a plain file: review.sh reads the full HEAD hash at the start, saves the verdict under .ai/reviews/ named by the first 12 characters, and writes the full hash on a Head: line. merge-check.sh refuses a missing file or a Head: line that names another commit, which covers your stale candidate row. Nothing covers your different signer row, because anything with write access to the repo can write under .ai/. review.sh also voids a review if HEAD moved while it ran, so when does your fixture read the candidate SHA, before the review starts or at signing?
Thanks, Iliya. In the fixture, the candidate SHA should be pinned before review starts—not first discovered at signing.
A trusted dispatcher captures full commit A and binds the run to it. The reviewer evaluates an isolated checkout of A, verifies the source identity (including any uncommitted changes) at the start and again before signing, and signs a receipt bound to A, the run, policy, and verdict. Signing never silently rebinds the review to a newer HEAD.
The merge gate independently resolves the current candidate (and, where relevant, base/merge result) and compares it with the authenticated receipt.
If the reviewed checkout changes during the run, no PASS is issued. If the branch advances from A to B but A remains an intact reviewed snapshot, the A receipt can remain evidence for A; it must not authorize B.
Your start/end HEAD checks cover an important race. The separate issue is producer authority: a matching
Head:line in a writable.ai/reviews/file is not an authenticated reviewer verdict.Our local lab tests the negative case with a different signing key and a pinned reviewer public key, but because both keys are created in one toy process, it does not prove real-world key custody.
The smallest additional test for your gate would keep HEAD=A, forge a PASS file with the correct full
Head: A, and requiremerge-check.shto reject it specifically for unauthenticated provenance. A separate race fixture would change the candidate during review and ensure no receipt is accepted for the changed candidate.That gives us two independently falsifiable properties: exact subject binding and reviewer-only issuance. Thanks for specifying where your gate already succeeds and where it does not.
I haven't run your forged PASS test, so what follows is the script's logic, with no run behind it.
merge-check.shasks for three things: a test pass record for the full HEAD hash, a verdict file for HEAD whoseHead:line matches, and an ACCEPT whose reviewer is a different engine from the builder. A hand-written file withHead: A, oneVerdict: ACCEPTline and another engine's name as reviewer meets every check on the verdict, so I expect it to pass, because the reviewer is only a name in a file. Your second test is covered: if HEAD moves or a file outside.ai/changes during the review,review.shvoids it and saves no verdict. Builder and reviewer both run on my machine, so where would you keep the reviewer's key so the builder can't sign with it?Thanks, Iliya. I checked
merge-check.sh, and your reading is correct: the verdict fields are checked, but reviewer provenance isn't authenticated.Your HEAD and working-tree checks already handle review-time drift.
On a shared machine with the same OS identity, I wouldn't claim the reviewer's private key is isolated from the builder.
For cryptographically independent approval, I'd use a separately controlled reviewer runtime or signing service, with the private key inaccessible to the builder. The merge gate would verify a signed receipt bound to the pre-pinned candidate SHA, policy, run identity, and verdict.
Without that trust boundary, the local gate remains a useful workflow guard, not independent authorization.
A forged ACCEPT fixture would be a useful next test.