Tool-Calling Design Patterns: Why Your Agent Picks the Wrong Function

Teams debugging a flaky agent almost always start in the same place: the system prompt. They rewrite instructions, add examples, tune temperature. Sometimes that helps. More often the real problem is upstream — the tool interface itself is ambiguous, and no amount of prompting fixes an ambiguous interface.

An LLM choosing between tools is doing something closer to API design review than reasoning. If two tools look similar, it will confuse them. If a parameter's purpose isn't obvious from its name and description, it will fill it in wrong. If a tool can fail in five ways but only returns one error shape, the agent will misdiagnose the failure and retry the wrong thing.

This post covers the tool-calling design patterns that actually move the needle on reliability, with runnable examples using the Anthropic Python SDK.

Why tool design matters more than prompt design

When you give a model a system prompt, you're writing internal documentation the model may or may not attend to closely. When you define a tool schema, you're writing an API contract the model uses at every single decision point — every turn, it re-reads every tool's name, description, and parameter schema to decide what to call.

That means tool schemas get more attention from the model than prose instructions buried in a long system prompt. It also means bad tool design compounds: one ambiguous tool doesn't just fail once, it fails every time that decision point comes up.

Three failure modes account for most of the tool-calling bugs teams hit in production:

Wrong tool selected — two tools overlap in purpose and the model can't tell them apart.
Wrong arguments filled in — the schema doesn't constrain values enough, so the model guesses.
Bad recovery after a tool error — the tool returns an error the model can't act on, so it retries blindly or gives up.

Each has a concrete fix.

Pattern 1: One job per tool, disjoint by design

The most common mistake is exposing two tools that do almost the same thing — search_orders and get_order_by_id, send_message and notify_user. The model has to guess which one you meant, and it guesses based on surface wording, not your intended semantics.

Bad:

tools = [
    {
        "name": "search_orders",
        "description": "Search for orders",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    },
    {
        "name": "find_order",
        "description": "Find an order matching a query",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    },
]

These are functionally identical from the model's point of view. Merge them, or make the split explicit and non-overlapping:

tools = [
    {
        "name": "get_order_by_id",
        "description": (
            "Fetch a single order by its exact order ID (format: ORD-XXXXXX). "
            "Use this only when the user has already provided an order ID."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string", "pattern": "^ORD-[0-9]{6}$"}
            },
            "required": ["order_id"],
        },
    },
    {
        "name": "search_orders_by_customer",
        "description": (
            "Search orders belonging to a customer when no order ID is known. "
            "Requires either an email or a phone number, not a free-text query."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "email": {"type": "string"},
                "phone": {"type": "string"},
            },
            "minProperties": 1,
        },
    },
]

The rule of thumb: if you can't describe the difference between two tools in one sentence that a new engineer would immediately understand, the model can't either. Collapse or clarify.

Pattern 2: Push validation into the schema, not the prompt

Every constraint you write in a description ("must be a valid ISO date", "should be one of: pending, shipped, delivered") is a constraint the model can and will violate under pressure. JSON Schema keywords are enforced structurally — the model is far more consistent about respecting enum, pattern, and format than prose.

{
    "name": "update_order_status",
    "description": "Update the status of an existing order.",
    "input_schema": {
        "type": "object",
        "properties": {
            "order_id": {"type": "string", "pattern": "^ORD-[0-9]{6}$"},
            "status": {
                "type": "string",
                "enum": ["pending", "shipped", "delivered", "cancelled"],
            },
            "effective_date": {
                "type": "string",
                "format": "date",
                "description": "ISO 8601 date, e.g. 2026-07-08",
            },
        },
        "required": ["order_id", "status"],
    },
}

If a value genuinely can't be constrained by schema (e.g. free-text notes), that's fine — but constrain everything that can be constrained. Every enum you add is one entire class of malformed calls that disappears.

Pattern 3: Design error returns for the model, not for a human

A tool that returns {"error": "Internal Server Error"} gives the model nothing to work with. It doesn't know whether to retry, ask the user for different input, or give up. Treat tool error responses as part of the interface, with the same care you'd give an API response body.

def get_order_by_id(order_id: str) -> dict:
    order = db.orders.get(order_id)
    if order is None:
        return {
            "error": "not_found",
            "message": f"No order exists with ID {order_id}.",
            "recoverable": True,
            "suggestion": "Confirm the order ID with the user, or use search_orders_by_customer.",
        }
    if order.status == "archived":
        return {
            "error": "archived",
            "message": f"Order {order_id} was archived and is read-only.",
            "recoverable": False,
        }
    return {"order": order.to_dict()}

The recoverable flag and suggestion field aren't for your logging pipeline — they're instructions to the model about what to do next. This is the single highest-leverage change teams make when debugging agents that "get stuck" after a tool call fails: the agent was never told failure was recoverable, or what to try instead.

Pattern 4: Keep tool count small, group with routing if you can't

Model tool-selection accuracy degrades as the tool count grows — not linearly, but noticeably past 15-20 tools in a single call, and sharply past 40-50. If your agent genuinely needs sixty tools, don't hand all sixty to every call.

Two practical mitigations:

Namespacing by domain: expose a small "router" tool first (select_domain: orders | billing | shipping), then load only that domain's tools for the next turn.
Retrieval over tool descriptions: embed tool descriptions and retrieve the top-k relevant tools per user turn, the same way you'd retrieve documents for RAG. Anthropic's own guidance on large tool sets recommends this once you're past ~50 tools.

def select_relevant_tools(user_message: str, all_tools: list[dict], k: int = 8) -> list[dict]:
    query_embedding = embed(user_message)
    scored = [
        (cosine_sim(query_embedding, embed(t["description"])), t)
        for t in all_tools
    ]
    scored.sort(key=lambda x: x[0], reverse=True)
    return [t for _, t in scored[:k]]

Pattern 5: Make idempotency explicit for anything with side effects

If a tool call can be retried — by the model, by your own retry logic, or by a flaky network — and it charges a card or sends an email twice, that's a design bug, not a model bug. Require an idempotency key for any tool with a side effect, and generate it yourself rather than asking the model to.

def charge_customer(order_id: str, amount_cents: int, idempotency_key: str) -> dict:
    existing = db.charges.get_by_idempotency_key(idempotency_key)
    if existing:
        return {"charge_id": existing.id, "status": "already_processed"}
    charge = payment_gateway.charge(order_id, amount_cents, key=idempotency_key)
    db.charges.save(charge, idempotency_key=idempotency_key)
    return {"charge_id": charge.id, "status": "created"}

Generate the key in your orchestration code per tool-call attempt (not per user turn), so a genuine retry of the same logical action reuses the key, but a new action gets a new one.

Comparison: symptom to fix

Symptom	Likely cause	Fix
Agent calls tool A when it meant tool B	Overlapping tool purposes	Merge or sharpen descriptions (Pattern 1)
Agent fills in a malformed status/date/ID	Constraint only in prose	Move constraint into `enum`/`pattern`/`format` (Pattern 2)
Agent loops or gives up after one tool error	Error response has no recovery signal	Add `recoverable` + `suggestion` fields (Pattern 3)
Tool accuracy drops as you add more tools	Too many tools in one call	Namespace or retrieve top-k tools (Pattern 4)
Duplicate charges/emails on retry	No idempotency key	Require a server-generated key (Pattern 5)

Common pitfalls

Writing descriptions for humans, not models. A README-style description ("This endpoint retrieves order data") is worse than a directive one ("Use this when the user provides an order ID; do not use for customer-wide search").
Reusing one tool for reads and writes. manage_order(action: "get" | "cancel" | "refund") looks efficient but forces the model to get both the action and the arguments right in one shot — split read and write tools so a wrong read can't accidentally trigger a write.
Not versioning tool schemas. If you change a required parameter, old prompts/examples in your few-shot library silently break. Treat tool schema changes like API breaking changes.
Testing only the happy path. Most tool-calling bugs show up in error handling and edge-case argument filling, not in the golden-path demo. Build eval cases specifically for malformed input and tool failures — see our guide to LLM evals for production agents for how to structure that suite.

Next steps

Start by auditing your current tool list for Pattern 1 violations — overlapping tools are the highest-frequency bug source and the cheapest to fix. Then move constraints out of descriptions and into schema, and add recoverable/suggestion fields to your error paths. Each of these is a same-day fix that measurably improves tool-selection accuracy without touching your prompt or your model.

If you're building agentic systems and want hands-on practice with these patterns — plus eval design, memory architecture, and production debugging — check out Joinloop's AI engineering cohort.