Low tool-call success? Fix these six layers before switching models

When an Agent has a low tool-call success rate, switching to a stronger model is the tempting first move.

Sometimes it helps. A stronger model may choose tools more accurately and produce difficult arguments more reliably. But our latest review found that many failures happened after the model had already responded: another validator rejected a valid tool name, the JSON Schema disagreed with runtime validation, the error result gave the model nothing useful to repair, a safe retry became a duplicate action, or a successful result disappeared before the next model turn.

A different model cannot repair those problems.

The main lesson is simple:

Tool-call success is not just a model metric. It measures the whole path from choosing a tool to using its result to finish the task. Fix the road before comparing drivers.

First, redefine success

Many dashboards count a call as successful when the model emits a tool_call. Real success needs a longer chain:

user task
  -> model selects the correct tool
  -> arguments satisfy the schema
  -> application accepts and executes the call
  -> result is paired with the original call_id
  -> result reaches the next model turn
  -> Agent uses it to complete the task

A well-formed call to a tool missing from the runtime is not a success. Neither is a completed execution whose result never reaches the model. A green tool badge cannot make an answer based on stale data correct.

We started by separating failures instead of replacing the model:

Stage	Typical failure
Selection	No call, wrong tool, or conflicting calls in one turn
Arguments	Missing field, wrong type, invalid enum, or invented optional value
Execution	Permission, dependency, domain, or side-effect failure
Protocol	Tool name, call ID, stream event, or result-pairing failure
Continuation	Lost or reordered result, or no clear next step
Task	Individual call succeeds, but the user goal remains unfinished

The layer tells you whether to change the schema, runtime, prompt, gateway, or model. Recording all six as “model tool failure” turns optimization into guesswork.

Layer 1: reduce the choice surface

An Agent may own many tools without showing every definition on every turn.

Large catalogues create similar names, competing descriptions, and heavy schemas. The differences that matter become harder to see. We now prefer a compact capability index and load only the tools relevant to the current task. Large installations can use Tool Search or their own progressive loader.

A tool description is not a product page. It should explain when to call the tool, where critical inputs come from, whether it has side effects, and what its result enables. Types, enums, and required fields belong in the schema. Repeating them in prose creates a second contract that can drift.

When a workflow must execute one step at a time, disable parallel tool calls. Parallelism is useful for independent reads. It is a poor fit for approvals, dependent steps, and side effects.

OpenAI’s function-calling guide treats Tool Search, strict schemas, tool choice, and parallel-call controls as separate mechanisms. They solve different problems; one instruction saying “call tools correctly” cannot replace them.

Layer 2: make schema and runtime tell one story

One recurring failure looked like model error but was not. The model generated arguments that matched the JSON Schema it received, while the application rejected the same payload because runtime validation followed another definition.

The safer design derives three things from one type source:

• the JSON Schema shown to the model;
• the runtime validator used by the server; and
• successful and failing fixtures used in tests.

Close objects when possible. State enums, bounds, lengths, and nested shapes precisely. properties means a field is allowed; only required means it must be supplied. When an optional value is unknown, omission is usually safer than an empty string, null, or a prose placeholder.

Where the provider supports it, compatible functions can use strict: true. Strict mode protects structure, not business truth. Server-side validation must still own permissions, inventory, balances, and current domain state.

Layer 3: make failures teach the model how to repair them

An early failure result looked like this:

{"ok": false, "error": "invalid_tool_call"}

It tells the model that something is wrong but not what to change. The next turn becomes another guess.

A bounded structured result is more useful:

{
  "ok": false,
  "error": {
    "code": "invalid_arguments",
    "message": "Please correct the rejected fields.",
    "issues": [
      {"path": "range.end", "code": "too_small", "message": "Must be later than range.start"}
    ]
  }
}

The code is stable, the path is precise, and the instruction is actionable. Raw private values, complete requests, stack traces, and internal exception messages should stay out of the model-facing result.

If exactly one tool fails argument validation and no side effect has started, the repair turn can expose only that tool and force it once with a named tool_choice. The model no longer has to choose a tool again; it only has to correct the form.

Repair still needs a limit. An infinite loop is not reliability with better branding.

Layer 4: separate safe retries from dangerous ones

Network timeouts, rate limits, and short gateway outages can be retried. The application first needs evidence that the previous attempt emitted no visible output and started no side effect.

Once text or a tool call has appeared—or a write may have executed—replaying the request can create duplicate messages, charges, notifications, or orders. The correct state is then “outcome unknown,” followed by an authoritative status check, not a blind retry.

A safer retry policy usually includes:

• retry only errors explicitly classified as transient;
• retry only before any model event or side effect;
• respect Retry-After and use bounded exponential backoff with jitter;
• distinguish connection, inactivity, and hard request deadlines;
• give writes idempotency keys and authoritative execution records; and
• fail clearly when the retry budget is exhausted.

Retries improve tolerance for temporary failures. They cannot repair a wrong tool name, conflicting schemas, or permanently invalid credentials.

Layer 5: give every call exactly one result

A multi-turn Agent depends on a strict timeline: the assistant emits a Tool Call, the application returns a Tool Result under the same call_id, and the model continues.

Browser reloads, stream disconnects, user interruption, approval waits, worker restarts, and duplicate events can all break that timeline. We ended up treating these as protocol invariants:

1. Every finished Tool Call has exactly one JSON result.
2. The result keeps the original call_id and chronological position.
3. Denial, interruption, and unknown outcome are distinct states.
4. Duplicate events may be idempotent; conflicting events must fail.
5. Session recovery never replays a side effect that may already have happened.
6. Tool definitions, stream parsing, approval policy, and execution dispatch share one registry.

That last rule catches an easy-to-miss class of failure. The model can select a real, implemented tool, only for an old event allowlist to reject its name. The dashboard reports low tool-call success, but the actual defect is an unsynchronized internal registry.

Layer 6: validate on the same tasks, then compare models

Only after the first five layers are stable does model comparison become meaningful.

We now separate at least these measurements:

• correct tool-selection rate;
• first-pass schema acceptance;
• actual execution success;
• success after one repair turn;
• model requests and tool calls per user task;
• final task-completion rate; and
• P50 and P95 latency, timeouts, rate limits, and retries.

Each trace should connect the user task, physical model request, Tool Call, Tool Result, and following model request. A trace is a travel log for a request. If it records only departure and arrival, it cannot explain the traffic jam in between.

The verification order matters too. Contract and unit tests should cover schemas, tool names, result pairing, retry boundaries, and side-effect protection. Next, replay a fixed task set. Finally, compare models under the same prompt, tool catalogue, schemas, gateway, and budget.

Otherwise, the candidate model receives a repaired road while the baseline model receives the old one. That experiment compares two systems, not two models.

When switching models really is the right move

Model choice still matters when:

• the model repeatedly chooses the wrong capability under the same tool set;
• arguments are structurally valid but semantically inconsistent;
• the workflow needs strict schemas, tool choice, context length, or streaming behavior the current model or provider does not support; or
• the improvement in task completion justifies the added latency and cost.

But when failures come from internal allowlists, duplicated schemas, lost results, unsafe retries, or mismatched call_id values, upgrading the model is just putting a better car on a broken road.

The real change in our latest optimization was not one magical prompt. It was the order of operations: define success, locate the failing layer, repair contracts and state, then let the model make decisions inside a smaller, clearer, repairable space.

Models matter. They should make judgments, not patch holes in the surrounding system every day.