Designing tools an LLM won't misuse

Originally published on Loop & Retry — field notes on building LLM agents that survive production.

Most agent bugs I've chased weren't in the model. They were in the tools — specifically, in the gap between what a tool's schema implied it wanted and what it actually did with what it got. The model is a caller that reads your parameter names and descriptions, forms a plausible theory of how the tool works, and acts on that theory under uncertainty. When it misuses a tool, the usual cause is that the tool let it.

You can't make the caller deterministic. You can make the tool hard to misuse. Four properties do most of the work: a legible schema, a validating boundary, recoverable errors, and idempotency. Here's each, with the failing version and the fix.

1. A legible schema: write for a reader who can't ask questions

The schema is the entire spec the model gets. It can't read your code, your docstrings elsewhere, or the ticket that explains the edge case. If the contract isn't in the name, the type, and the description, it doesn't exist. Here's a tool that leaks its contract:

# BAD: what does any of this mean, and what's allowed?
{
    "name": "search",
    "description": "Search for items.",
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string"},
            "filters": {"type": "string"},   # a string of... what?
            "options": {"type": "object"},   # anything goes
            "limit": {"type": "integer"},
        },
        "required": ["query"],
    },
}

Every field here invites a guess. filters is a string, so the model will invent a syntax — "status:open", or "status=open,priority=high", or JSON, depending on its mood — and you'll parse whichever it picked. options is a free object, which means the model can pass anything and you handle nothing reliably. limit has no bounds, so you'll eventually get limit: 10000. The name search doesn't say search what.

The fix is to make illegal states unrepresentable in the schema itself, and to spend words on the description where the type can't carry the meaning:

# GOOD: the schema is the spec; enums close off invention; ranges bound blast radius
{
    "name": "search_support_tickets",
    "description": (
        "Search the customer support ticket database. Returns tickets ordered "
        "by last-updated, newest first. Use `status` and `assignee_email` to "
        "narrow results; omit them to search all tickets. Does NOT search "
        "archived tickets older than 90 days — use `search_ticket_archive` for those."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "Free-text search over ticket subject and body.",
            },
            "status": {
                "type": "string",
                "enum": ["open", "pending", "resolved", "closed"],
                "description": "Filter to one status. Omit to include all statuses.",
            },
            "assignee_email": {
                "type": "string",
                "description": "Filter to tickets assigned to this exact email address.",
            },
            "limit": {
                "type": "integer",
                "minimum": 1,
                "maximum": 50,
                "description": "Max results to return (1-50). Default 20.",
            },
        },
        "required": ["query"],
    },
}

What changed, and why each matters:

• The name states the object. search_support_tickets, not search. When an agent has fifteen tools, a bare search competes with search_docs and search_users for the same intent, and the model picks wrong. Name the noun.
• enum replaces a free string. The model literally cannot pass an invalid status now; the set of legal values is in front of it. Every free-text field where the real domain is a fixed set is a future parsing bug.
• minimum/maximum bound the blast radius. A model that asks for 50 results when it wanted "a few" is a mild inefficiency. One that asks for 10,000 because nothing stopped it is an incident.
• The description carries the contract the types can't — ordering, defaults, and crucially the boundary ("does NOT search archived tickets, use this other tool"). Telling the model where a tool's responsibility ends is how you stop it from forcing the wrong tool at a problem.

Keep the surface small, too. Every optional parameter is another axis the model can get wrong. If you have a tool with twelve optional knobs, you probably have three or four tools wearing a trench coat — split them by intent so each call has an obvious shape.

Continue reading: the rest of this post — including the sections that follow — is on Loop & Retry: read the full article