An agent contract requires retrieve_policy. The trace seems to contain retrieve_policy. The evaluator says the tool is missing.
Before changing the contract, inspect the characters.
const expected = "retrieve_policy";
const observed = "retrieve_p\u043Elicy";
console.log(expected === observed); // false
The observed value contains a Cyrillic small letter o, U+043E, where the expected identifier contains Latin o, U+006F. In many fonts, the names look alike. They are different strings.
I maintain AgentInspect. This example was tested against 6.31.35, whose diagnostics can expose code points for certain suspicious names. The important design choice is not a clever Unicode trick: improve the explanation without silently changing the identifier being evaluated.
Equality and readability solve different problems
A tool name participates in a contract between a tool registry, an invocation, a trace adapter, and an evaluator. If those components disagree, forgiving display logic can hide a real integration defect.
| Operation | Appropriate purpose | Risk if used for policy matching |
|---|---|---|
| Exact comparison | Compare recorded identifiers | May need better diagnostics for humans |
| Escaped/code-point display | Explain visually ambiguous values | Must not replace the raw identity |
| Friendly display label | Make a UI readable | Several identifiers may share a label |
| Case folding or normalization | Explicitly designed naming schemes | Can silently merge names under a different scheme |
An allowlist is not a spellchecker. If the application authorizes one identifier, an adapter should not authorize a lookalike on its behalf.
This does not mean non-Latin tool names are invalid. The contract can explicitly allow a name in any supported naming scheme. The problem is disagreement between identities, not the language used to write them.
Preserve identity; make the mismatch legible.
Reproduce the mismatch without an LLM
Install the pinned package and capture a synthetic invocation:
npm install --save-exact agent-inspect@6.31.35
import { inspectRun, step } from "agent-inspect";
await inspectRun("unicode-name-fixture", async () => {
return step.tool("retrieve_p\u043Elicy", async () => ({
found: true,
}));
}, { traceDir: ".agent-inspect", silent: true });
The callback succeeds. That does not make the name match the contract. After opening the resulting trace with openTraceFile, evaluate:
import {
defineTraceContract,
evaluateTraceContractRead,
} from "agent-inspect/checks";
const contract = defineTraceContract({
tools: {
required: ["retrieve_policy"],
allowed: ["retrieve_policy"],
},
});
const result = evaluateTraceContractRead(trace, contract);
console.log(result.status, result.findings);
The measured fixture fails. Its findings include the U+043E diagnostic. A positive control using the exact ASCII name passes.
The two constraints are complementary. required asks whether the expected tool occurred. allowed asks whether an unexpected tool occurred. Depending on your application, the meaningful report may include both a missing expected operation and an unrecognized observed one.
TraceContract remains a beta API. Keep the regression fixture pinned to the version used by your release gate.
Trace the mismatch back to its boundary
When a name differs, investigate the registry, serialized invocation, adapter mapping, and contract definition. A copied description may have become an identifier. A display label may have replaced a registered name. A normalization layer may have rewritten characters.
Do not assume an attack from this fixture alone. Copy/paste errors, inconsistent naming conventions, and adapter defects can produce the same symptom. Diagnostics should report the observed mismatch without making an unsupported claim about intent.
The same discipline applies to leading spaces, case changes, hidden controls, and aliases. First establish the application's naming contract. Then determine which boundary violated it.
If aliases are intentional, declare them in a reviewed mapping and version it. An undocumented “close enough” conversion makes later evidence harder to interpret.
Reduce opportunities for drift
For names you control, share constants between registration, invocation, and test expectations:
export const Tool = {
retrievePolicy: "retrieve_policy",
summarizePolicy: "summarize_policy",
} as const;
type ToolName = typeof Tool[keyof typeof Tool];
function acceptsRegisteredName(name: ToolName) {
return name;
}
The type prevents certain accidental literals in your own TypeScript code. It does not validate an untrusted runtime string. Validate external values at the boundary using the registry's actual identifiers.
Also separate identifiers from descriptions. A readable label can change during localization without renaming an operation. Conversely, changing the operation's identifier should trigger a deliberate migration of contracts and historical comparisons.
Keep diagnostic output bounded
A helpful error report shows the expected name, the observed name, and enough escaped or code-point information to explain the difference. It should not dump unbounded input into a terminal or report.
The current implementation applies additional code-point detail to selected mixed-script or risky-control cases; it is not a universal spoofing detector. A name that receives no extra diagnostic is not thereby trustworthy. Likewise, a name receiving a diagnostic is not automatically malicious.
Keep the raw captured evidence, the comparison result, and the human-friendly explanation conceptually separate. Each has a different job.
Don’t repair the evidence to make the test green
The tempting fix is to normalize the observed name into the expected one. That may make a fixture pass while hiding the actual mismatch.
A better fix corrects the naming boundary or explicitly updates the contract. Then rerun both the exact-match positive control and the lookalike negative control.
When an agent tool crosses registries, SDKs, adapters, and evaluators, a tiny identifier change can become a large debugging detour. The solution is exact policy and understandable diagnostics—not a more forgiving guess.
Implementation references
- Tool matching and bounded diagnostic formatting
- Contract documentation
- Companion:
examples/probes.test.mjs, testD04.

Top comments (2)
Worth adding the trap that catches people trying to fix this with normalization:
"retrieve_p\u043Elicy".normalize("NFKC").lengthstays the same and still !== the Latin version, because U+043E is a distinct Cyrillic letter, not a compatibility character. NFKC won't fold it, and neither will toLowerCase. So a "just normalize both sides" patch passes code review and changes nothing.The check that does work is a confusable skeleton: map every name through UTS #39
skeleton()and compare that, while keeping the raw string for the actual dispatch. Cheap version without a library: flag any identifier that mixes scripts, e.g./\p{Script=Latin}/u.test(n) && /\p{Script=Cyrillic}/u.test(n), and reject that at registry time with the offending code points printed. A mixed-script identifier is almost never intentional.One question: does AgentInspect compare the trace's tool name against the registry after JSON-decoding, or before? A
\u043Eescape inside the recorded JSON is harmless once decoded, so flagging pre-decode would produce false positives on serialized traces."Seems to contain" is the dangerous part. Fuzzy matching tool names means an agent can claim it called the policy check when it called something with a similar name. Exact identifiers are the only safe comparison.