How AI agents learn to use tools—and how to test them

Diyi Yang introduces a lecture on why agents need tools, how models learn to select and use them, and how tools are exposed through interfaces. She closes with benchmarks and reliability measures for tool use.

She first revisits retrieval-augmented generation (RAG): retrieve external material and use it to answer or perform a task. Even this design can fail at retrieval, reading and the interaction between stages.

Retrieval can expose material the agent should not access. A reader can be distracted by relevant-looking documents or face conflicting evidence; an early error can also propagate through the retrieve-read-generate loop.

Prompt injection adds another risk: an adversary can place malicious instructions in retrieved documents. Yang says a system must treat retrieved material carefully rather than assume it is trustworthy.

To illustrate “lost in the middle,” she describes a question-answering prompt with several documents, one containing the answer. She asks whether a model uses the answer equally well when its document moves from the beginning to the middle or end.

In the classic study she presents, performance often forms a U-shape: relevant material near the beginning or end is used more reliably than material in the middle. The correct answer can be present in context and still be ignored.

Yang illustrates the effect with an older question-answering example: the displayed closed-book score is about 56, while supplying a relevant document can raise it to 88. In the middle position, performance can fall back toward the no-retrieval baseline.

Retrieving more documents increases the likelihood that an answer is present, but does not guarantee the reader will use it. In the figure, retrieval reaches about 90% around 50 documents while reader accuracy levels off earlier; Yang recommends choosing and reranking results carefully.

Citations help make answers checkable, but generated citations can themselves be wrong. Yang says fluent, apparently authoritative responses should not be trusted solely because they contain links or references.

She shows an older comparison in which users rated answers highly while citation precision was substantially lower. The contrast illustrates that perceived usefulness and factual support are different measurements.

Yang says researchers have found hallucinated references in conference papers and student work, and describes a conference response involving desk rejection. She urges students to verify references and take responsibility for what they submit.

She warns that a link or citation can make an answer look authoritative without verifying its contents. The submitter remains responsible for checking the sources.

A needle-in-a-haystack test places one relevant fact in a long context and varies its location and the length of surrounding text. Yang presents older Claude 2.1 results showing more failures as context grows, especially away from the most recent information.

In this test, context length and the position of the inserted fact are varied independently. The model then answers a question whose answer depends on that fact.

An older GPT-4 test looks stronger overall, but Yang points to degraded regions at longer lengths and says middle-position failures still appear. Her lesson is that a large advertised context window does not guarantee reliable use of every fact within it.

Yang emphasizes that these examples concern particular earlier models and tests. Their lesson is to measure actual retrieval and reading behavior rather than infer it from context-window size.

Asked why the position effect occurs, Yang offers long-range dependencies and positional biases as possible explanations. She treats them as intuition for the observed behavior rather than a complete proof of cause.

She compares attention to recent information with a human tendency to remember yesterday more readily than an earlier event. She presents this as an analogy for position bias.

A student asks whether retrieval has alternatives for external knowledge. Yang still recommends RAG as a useful general approach, coupled with checks for correctness, position bias, access permissions and injected content.

For a particular application, she recommends checking whether the system gives correct answers without being unduly affected by document order. Security checks should also address injected material.

Yang turns to tools with older model examples: a model can struggle with current dates, live facts or arithmetic even while succeeding on harder-looking problems. She shows multiplication accuracy deteriorating as the number of digits increases.

Yang uses multiplication to make the contrast concrete: earlier models could solve some sophisticated problems yet make mistakes on straightforward exact arithmetic. A calculator can provide an independent operation.

Similar trends appear in examples involving puzzles and dynamic programming: more steps can make a task harder for the model. Yang proposes delegating exact computation to external tools rather than assuming the language model must perform every operation internally.

She says performance tends to worsen when a problem requires more intermediate steps. This motivates delegating operations to specialized software rather than relying solely on fluent text generation.

She groups limitations into exact reasoning, access to information absent from training, and interaction with non-language systems or the physical world. Calculators and code can help with computation; retrieval and live search provide changing or private information.

A live API can provide current information, while retrieval over company documents can provide material missing from model training. Different external tools address different limitations.

Other tools can expose databases, browsers, APIs, image generators or robots. Yang treats tool use as a broad agent capability: the system chooses an external operation and uses its returned result in the task.

A tool can be viewed as an interface to a program outside the model. The model produces a name and arguments in text; the external program performs the operation and returns a result.

The textual interface matters because language models already handle text. A time function, calculator or weather service can be represented by a recognizable name and specified parameters, even though execution happens elsewhere.

Yang expands the examples to calendars, email, image generation, translation systems and dictionaries. A tool need not be a new model: an existing service or specialized model can provide a capability the agent lacks.

The tool catalog can include conventional services and specialized models, such as a translation system. The agent does not have to reproduce each capability internally.

For a multiplication problem, the model emits a calculator call with the operands; a weather question can trigger a weather function. In both cases the model incorporates an external answer rather than generating the result unaided.

The tool call is represented in a structured text form with an operation and inputs. An external system performs the work and passes its output back to the model.

An audience member asks how text-based calls cope with type checking, versioning and changing meaning. Yang says generated names or arguments can be hallucinated or invalid, so external checks can fail; these are real tool-use failure modes.

Another question asks whether models can create tools dynamically. Yang says research has explored generating a tool or interface on the fly, but the generated tool can itself contain errors or reproduce something the model has memorized.

The lecture then asks how a model learns when to invoke a tool, which tool to choose and which arguments to provide. Yang introduces Toolformer as an early approach that makes these decisions a learning problem.

Toolformer’s setup augments ordinary text with calls, their results and subsequent text. Yang uses a calculation and a factual question to illustrate a possible insertion point where an external answer would improve the next words.

The intended behavior includes deciding the location of a call, selecting the appropriate API, creating its arguments and using the returned result. Yang treats each as part of the learning problem.

The early system used five tools: question answering, a calculator, search, translation and a calendar. Training on augmented examples can teach a model to emit a tool call and incorporate its result at inference time.

Ordinary sentences are augmented with candidate API calls to build training examples. At inference, the trained model can emit the special call format while generating text.

To construct examples, the method considers positions in a sentence where an API call is likely to be useful. It then samples candidate calls at those positions and executes them externally.

After choosing an insertion point, the procedure samples possible calls and executes them. That produces candidate responses whose usefulness can be compared.

The filtering step asks whether a returned result improves prediction of the text that follows. Its weighted loss places more emphasis on nearby tokens, so the immediate effect of a tool result matters more than distant words.

The loss comparison includes a baseline that supplies the call without its result. This tests whether the returned information adds value beyond the call text itself.

Yang compares the loss with a complete call and result against baselines with no call or a call without its output. If the result improves prediction by more than a threshold, the candidate can be retained for training.

In her Pittsburgh example, the displayed losses are 2.5 without a call, 2.1 with a call but no answer, and 0.8 with the executed answer. The improvement against the stronger baseline is 1.3; lower loss is better.

An audience member asks how reliable this is when the external service may return cached data or fail. Yang agrees that unavailable servers, malformed requests and bad responses affect the training signal, illustrating limits of the learning-based approach.

Yang notes that an external service can be unavailable or return an error even when a candidate call looks sensible. The procedure evaluates what actually comes back, which can limit the resulting signal.

She shows examples where a useful calendar call receives a higher score than an unhelpful call. The score is meant to identify whether a particular tool response helps at a particular point in the text.

Yang says Toolformer’s small GPT-J-based model performed well on the tasks shown, including mathematics and temporal information, compared with much larger baselines. The historical result demonstrates that access to tools can compensate for some model limitations.

A student asks whether a tool call happens inside token generation or between agent turns. Yang says the early example inserts a call into the generated stream; newer agents can produce a longer trace, try one tool and then choose another.

The distinction matters historically: an inserted call in a token stream is not the only way to organize an agent. A later system may explicitly plan, inspect a result and choose another action.

ToolkenGPT takes a related token-oriented approach: represent each tool with a learned token embedding. When that token is selected, generation pauses, the model supplies arguments, the external tool runs and the returned result is fed back before generation continues.

Yang illustrates the mechanism with a greatest-common-divisor function in a math problem. The system alternates between ordinary reasoning text and a tool mode rather than calculating the entire answer in one uninterrupted stream.

Synthetic examples can train tool embeddings, but Yang highlights a historical weakness: tasks requiring several calls perform much worse than one-call tasks in the displayed results. She says multi-step tool use remains harder than selecting a single operation.

Asked why, Yang points to the large search space of possible call sequences and relatively limited examples of multi-step tool use in training data. She identifies the combinatorial search problem as the major reason in her explanation.

A student asks how a model discovers newly created tools. Yang uses that question to transition from a fixed handful of tools to methods intended to scale across thousands of APIs.

ToolLLM gathers a large collection of API descriptions and uses a capable model to synthesize plausible user requests and solution traces. Yang says the system covers more than 16,000 APIs in the collection she presents.

The process prompts ChatGPT to generate a user request for a tool, then to attempt the task with available tools. The resulting solution paths form the ToolBench dataset; a smaller Llama model can be fine-tuned on those examples to create ToolLLaMA.

At runtime ToolLLaMA receives an instruction together with candidate APIs and produces an answer using its learned tool-use behavior. The training examples were generated from simulated requests and solution paths.

Yang distinguishes training a smaller model on generated task examples from classical knowledge distillation. The examples model specific user requests and solution steps; they do not require access to another model’s private internal reasoning.

Yang stresses that generating a specific synthetic task is different from extracting a model’s hidden reasoning. The resulting examples can still be used to train another model.

She shows tool-call failures for speech-to-text: one example invents parameters, while another selects a translation tool for a transcription request. Similar names and interfaces make selecting the correct API and arguments difficult.

A call can fail in several ways: an API might be invented, parameters might not exist, or the selected function might address the wrong task. Yang uses transcription versus translation to illustrate the distinction.

Gorilla is another system for connecting language models to APIs. Yang describes a collection of about 1,600 APIs from model hubs and synthetic instructions generated with GPT-4, followed by model training.

She describes retrieval-aware training that exposes the model to correct and incorrect documentation. The goal is to make it more robust when retrieved documentation is not the right match, rather than merely copying whatever appears in context.

At inference, a user request is matched against the available API descriptions and the system constructs a call. Yang notes that users may see only the final result even though the intermediate selection and calls consume tokens and cost.

Exact string matching can be too brittle for API evaluation. The approach she presents parses a call into an abstract syntax tree and compares its structure with a reference, helping distinguish a nonexistent API from other malformed calls.

A syntax tree makes the structure of a proposed call comparable with a reference. Yang uses this to discuss both invalid API names and errors within an otherwise recognizable call.

Yang shows results in which Gorilla has lower measured API hallucination and higher accuracy than the compared baselines. Structural validity is one part of evaluation; an executable-looking call is not by itself evidence that a task was completed correctly.

These training approaches create a maintenance problem: newly introduced APIs can require additional training or updated tool representations. Yang shifts to describing tools consistently at runtime so models can discover them through a shared interface.

A shared runtime description changes the integration problem: the model can read available tool descriptions instead of having each tool permanently encoded through retraining. That does not remove the need to test its choices.

She compares three interface patterns: REST requests, structured function calls and the Model Context Protocol (MCP). The shift is from adapting a model to a fixed tool collection toward making services discoverable to applications.

Yang describes REST, function calling and MCP as different ways to connect applications and services. They expose information at different levels rather than solving every integration issue automatically.

A REST example sends an HTTP request to a weather endpoint with a location parameter and reads the response. The caller must know the endpoint, parameter names and response format, among other details.

Function calling exposes a tool with a structured schema and asks the model for a name and typed arguments. Application code can validate the request, execute the function and return the result; the schema still has to be maintained as tools change.

The application around a function call can inspect whether the emitted arguments conform to the expected schema before executing it. A schema describes the call; it does not guarantee the model selected the right function.

Yang presents MCP as a common protocol between AI applications and tool or data providers. A server exposes capabilities and a client connection lets a host discover and invoke them at runtime, allowing integrations to be reused across compatible applications.

In the terminology she uses, the host is the application, a server provides access to resources or tools, and a client is a connection the host establishes to a server. Several connections do not imply several human users.

A host can request a server’s tool list, which includes names, descriptions and input schemas, then invoke a chosen tool with arguments. The model still has to select the appropriate operation and supply valid inputs.

Yang says developers can use libraries to express higher-level tool specifications rather than manually write every protocol message. Her example wraps course services for agent access.

For a course learning-management system, Yang imagines tools to list assignments, read submissions or post grades. Those capabilities require different permissions: merely connecting an agent must not give every user authority to change grades.

In her example, an MCP server sits between the agent and the course API and checks access. She describes keeping the course API token separate from the agent’s token, so the integration can enforce what each caller is permitted to do.

Yang introduces the Berkeley Function Calling Leaderboard as a source of tool-use evaluations. She describes checking call structure, actual execution and final state, including examples where the right answer is that no suitable tool is available.

She revisits ToolBench and then describes τ-bench, where an agent interacts with a simulated customer—for example, to change or cancel a flight—and must use tools in accordance with the task.

For multi-turn interactions, Yang distinguishes pass@k, where at least one of k attempts succeeds, from a stricter reliability measure, pass-hat@k, which asks whether all selected attempts succeed. A user needs a system that works consistently, not just once in many tries.

She notes that τ-bench has expanded beyond its earlier retail and airline scenarios. Yang closes by emphasizing that the evaluation metric affects what a benchmark says about reliability.