Platform teams already enforce fail-safe defaults for Kubernetes objects with RBAC and admission controllers such as OPA/Gatekeeper and Kyverno. My preprint, Fail-Closed Authorization for Tool-Using AI Agents: A Capability Contract and Case Study, asks what the same kind of admission check should look like for agent tool calls, and what a plain tool-level allowlist leaves uncovered. It describes the Agent Security Contract (ASC), a deny-by-default gate that decides on each agent or MCP tool call before it executes. If the policy does not explicitly allow a call, the call does not run.
It is a preprint and has not been peer-reviewed. The paper itself calls it a systems/engineering contribution rather than a new enforcement mechanism, and closely related enforcers exist.
What the contract contains
An ASC policy is a YAML file with:
- a capability catalog with a pinned
capability_digest(a SHA-256 digest of the approved tool catalog) - per-agent allowed tools, each with argument constraints: type, regex
pattern,deny_patterns,allow_hosts,max_length - a list of high-impact tools that need out-of-band approval
- rate limits per trace and per tool
The demo policy in the paper declares 8 capabilities, allowlists 7 of them for demo-agent (secrets.read is left out) and marks filesystem.delete and deploy.production as high impact.
How a decision is made
Every decision is one of ALLOW, BLOCK or REQUIRE_APPROVAL. Only ALLOW reaches the tool. BLOCK and REQUIRE_APPROVAL go back to the agent.
Evaluation is ordered and short-circuits. The first three checks in the paper:
- Unknown
agent_id: block. - Catalog digest mismatch: block.
- Rate limit exceeded: block.
The catalog digest is SHA-256 over the list of {name, description, input_schema} objects, sorted by name and serialized with sort_keys=True, separators (",",":") and ensure_ascii=False, UTF-8 encoded. No Unicode normalization is applied, so two descriptions that look the same but are encoded differently give different digests and the call is blocked. Other catalog fields, such as annotations, are not covered.
The fail-closed part goes beyond the default verdict. Every error path the paper enumerates and tests resolves to BLOCK, so an error inside the gate does not turn into an allowed call.
ASC runs in two modes. At runtime, the agent runtime calls the gate (as a library or a loopback HTTP sidecar) once per proposed call, passing the calls already admitted in that episode plus the new one. In CI/admission mode, a recorded trace is evaluated as a whole, as a regression or admission check. All results in the paper use the second mode.
Evidence for every verdict
Each decision is written as a JSON evidence envelope. According to the repo's schema it carries schema_version, policy_version, policy_sha256, scenario_id, verdict, reason, trace_id, commit_sha, timestamp and envelope_sha256, a SHA-256 over the canonical JSON of the other fields. asc verify rejects envelopes with missing fields, inconsistent verdicts or edits that no longer match the digest, and with --policy it also rejects a policy_sha256 that differs from the policy file.
The digest is unkeyed. It catches accidental or naive edits, but anyone who can rewrite the file can recompute it, so it is an integrity check and not a signature. Envelopes are trustworthy only when they come from a CI run or a pipeline you control.
From verdict to audit record
An envelope records the decision. It doesn't say which workload ran the call, on whose behalf, or what the model step cost. I wrote a separate guide on that part, AI agent audit trail: what to log for every tool call. It is a design for an audit trail on Kubernetes, not a product.
The starting point is that the logs you already have each miss something. Kubernetes audit logs only see requests to the API server, so an agent calling send_email through an MCP server never shows up there. Application logs usually record the tool call but not which policy allowed it. LLM traces often drop arguments and results for privacy. Gateway logs have tokens and cost per request but not the tool call the request led to.
The guide proposes one record per tool call, blocked and approval-pending calls included, with field groups for who (agent_id, workload_identity, on_behalf_of), what (tool_name, tool_catalog_digest, arguments_digest), the decision (verdict, reason, policy_version, policy_sha256, approval fields), outcome, cost, correlation IDs (trace_id, tool_call_id, mcp_session_id), provenance and a record_digest. A missing approval on a call that did not need one is information. A missing policy digest on any call is a gap.
Some rules from the guide apply directly to the gate.
- Use the OpenTelemetry GenAI names where they exist, such as the
execute_toolspan andgen_ai.tool.name. These conventions are at Development stability, so pin the semconv version you follow. - Leave
gen_ai.tool.call.argumentsandgen_ai.tool.call.resultoff by default (they are Opt-In) and store a digest plus a redacted copy instead. - Bind approvals to the argument digest. An approval for
delete /workspace/tmp/report.jsonmust not coverdelete /workspace. - Write records to append-only storage the agent's identity cannot modify, chain the digests so removed records are detectable, and add signatures if third parties rely on the records. An unkeyed hash is not enough there.
The guide's own test is short: if you cannot reconstruct identity, policy version, verdict and cost for a single tool call, you have logs and not an audit trail.
The case study
The open-source implementation was evaluated on static, recorded tool-call traces only. No live agent was run. All numbers below are copied from the abstract, each with its scope.
- Adversarial probe, 60 cases. Bypasses of the v0.1.0 demo policy found in review were fixed in v0.2.0. Allowed attacks fall from 32 to 1 (a policy gap), at the cost of 3 new benign over-blocks.
- Author corpus, 66 scenarios (v0.2.1). ASC returns the expected verdict in 66/66 cases. I built this corpus around these checks.
- Tool-name RBAC baseline (Python and OPA/Rego). It admits 30 of its 33 attacks, by construction.
- Mutation testing. Only 8 of 62 single-constraint mutants are killed, which means the corpus exercises few policy constraints.
- Held-out AgentDojo replay. 132 AgentDojo v1.2 ground-truth traces, replayed under policies frozen beforehand. With tool-level approval policies, 25 of 26 non-empty attack traces are held for approval, and so are 60 of 97 benign tasks. v0.2.0's strict-argument rule makes these unadapted policies block 94 of 97 benign tasks.
- Policies derived from AgentDojo's tool signatures. They first hit v0.2.0's lack of optional arguments (31 benign blocks). With v0.2.1 they reproduce the tool-level verdicts on all 132 traces, with no blocks and no newly admitted attack. Type-only declarations add no protection here.
- Latency. Median in-process latency is about 38 µs per trace.
Limits
These are taken from the paper's threat model.
- The
approvedflag is a boolean field of the call inside the trace in all released versions (v0.1.0 to v0.2.1). ASC does not authenticate it. A runtime that lets the model fill it in voids approval gating. Binding approvals to an authenticated principal is listed as future work. - Catalog pinning only detects a changed catalog, including a change after approval, if the runtime passes the catalog actually shown to the model on every call. A stale or missing catalog defeats the check.
- Out of scope: compromise of the evaluator host, CI or approval channel; envelope forgery by someone with write access; attacks that fit entirely inside allowed calls and arguments, such as exfiltration through an allowlisted host; semantic misuse no syntactic constraint captures; OS-level isolation.
- ASC is not a prompt-injection detector, a sandbox or a signer. It is meant to sit next to red-teaming, regression harnesses, runtime guardians and model-level defences.
The security goal is stated narrowly: no proposed call outside the declared contract executes, and any high-impact call without approval is held. A permissive constraint admits whatever it permits.
Links
- Research page: https://justrunme.com/research/fail-closed-authorization-ai-agents/
- Zenodo: https://doi.org/10.5281/zenodo.24075886
- Research Square: https://doi.org/10.21203/rs.3.rs-11676427/v1
- Code: https://github.com/justrunme/agent-security-contract
- Related guide: https://justrunme.com/guides/ai-agent-audit-trail/
About the author
Andrey Lesnikov is a Senior DevOps & AI Platform Engineer and an independent researcher. He writes about access control and audit trails for AI agent tool calls.
Links: justrunme.com · GitHub · ORCID · Academia.edu · Sessionize · Hashnode · Bluesky · Amazon author page
Top comments (4)
An argument
patternthe model can also see does a second job: it shapes what the model writes, and that moves the gate's numbers.In one eval suite,
send_emailholds the recipient to^[A-Za-z0-9._%+-]+@acme\.example$, and llama-server turns that schema into a grammar as the model writes. Pattern shown, Granite-4.0-1B tried outside 0 times and sent 16 emails to colleagues nobody asked for. Hidden, 37 attempts outside and 2 such emails. 13 of the 16 kept the outside name on the company's domain:bob.k@gmail.examplearrived asbob.k@acme.example.Each one passes a host allowlist. So a zero in
BLOCKcan be the decoder, not the policy. Running every model both ways is what split the two (ADR 13).The 8-of-62 mutation result is a useful caveat alongside the 66/66 author corpus; targeted mutants for each policy constraint could make that coverage gap easier to interpret. Are you planning to expand the corpus?
"Bind approvals to the argument digest" is the rule I'd underline. An approval that doesn't name exactly what it approved quietly turns into a blanket pass.
Disclosure: I build CutVoke, an open-source local video editor that Claude Code / Codex drive over MCP, so my stakes are much lower than deploy.production, but we hit a cousin of the same problem: a human looks at a proposed edit, and by the time it's applied the project has moved. What we ended up with: every edit goes through one
command_applytool that requires anexpectedRevision, and a stale revision is rejected as a conflict instead of being applied on top of whatever changed. There's also acommand_previewthat dry-runs an edit on the current revision without writing anything, and it refuses commands with outside side effects (export, audio processing, job creation), so "preview" can never accidentally do the real thing.It's not an authorization gate in your sense: no policy, no digest of the tool catalog. But the habit carries over: what was reviewed should be pinned to a version, and a version mismatch should fail closed.
Curious whether ASC's REQUIRE_APPROVAL path records which call arguments the approval covered, or whether that's left to the runtime.
The distinction between a policy verdict and evidence of actual enforcement is particularly important here.
One negative test I'd add is this: the gate returns REQUIRE_APPROVAL, but the runtime proceeds to invoke the tool anyway because it treats the verdict as advisory.
In our work on Agent Role Contracts (ARC) and a separate runtime guardian, we're exploring how to make that boundary independently testable.
The expected result should be not only a BLOCK verdict, but observable evidence that no downstream invocation occurred.
Since your published results are based on trace replay, are you planning a live adapter test that verifies this no-forward property under approval races or audit-write failures?