TECHNOLOGY EXPLAINER
How AI Agents Work
— what a language model can do once it is given tools and a loop, and where it breaks
An "AI agent" is a mechanism in which a language model uses "tools" such as search, calculation, code execution or operation of internal systems, and keeps choosing actions while looking at the results to carry a task forward. It sounds convenient, but "how far can it be trusted?" and "what happens when it fails?" cannot be judged without knowing how it works. This article sets out, from primary sources, the basic structure of LLM + tools + a loop, function calling, planning and memory, the difference between workflows and agents, what is actually being automated, the types of failure, and how to read the evaluations and standards.
- What an AI agent is (the short version)
- Workflows versus agents
- The basic structure — LLM + tools + a loop
- Planning and memory
- Workflow patterns, and multiple agents
- What is actually being automated
- Types of failure (with our calculation)
- Limits, and what could not be confirmed
- How to read evaluations and standards
- Glossary
- References (primary sources)
- Claim-to-source audit
Sourced = stated in published material (link given)
Our calculation = a figure this article derived, with the assumptions spelled out
Not yet confirmed = a plan, schedule or draft whose outcome has not been confirmed
Other readings are marked separately as Commentary. Inference is shown in bold.
This field moves fast, and tools and forms of offering change within months. The dates and numbers here are what could be confirmed in public documents on 15 October 2026. Benchmark numbers are values for the models and settings at the time of each paper or report, not the performance of current models. Numbers from different settings or evaluators are not lined up to rank anything. Numbers a developer gives about its own product are treated as "the developer's statement".
1. What an AI agent is (the short version)
- The basic idea is "an LLM uses tools and proceeds in a loop, looking at results from the environment": there is no magic. The language model's output (which tool to call, with what arguments) is executed by the application and the result is returned, and this round trip repeatsSourced [Sources 1, 4, 5]
- A "workflow" and an "agent" are different: in a workflow, code decides the path; in an agent, the LLM decides its own process and how to use tools. For many uses, a simple workflow or a single LLM call is enoughSourced [Source 1]
- The longer the work, the faster the chance of failure rises: even if each step succeeds 99% of the time, the chance of succeeding at 50 steps in a row is about 61%. What has been automated has to be read together with the length of the task, the conditions and the number of trialsOur calculation / Sourced
Judge an agent not by its "average success rate" but by "does it succeed every time when the same task is repeated" and "what can happen when it fails (permissions, and how easily the action can be undone)". The τ-bench paper introduced a metric (pass^k) for reliability over repeated trials of the same task, and reported that the state-of-the-art function-calling agents of the time succeeded on fewer than half of the tasks and, in the retail domain, succeeded every time in eight repeats on fewer than 25% of tasksSourced [Source 13].
2. Workflows versus agents
The definition of "agent" differs from document to document. Anthropic calls everything from fully autonomous systems to those that follow predefined steps agentic systems, and then draws a structural distinction. A workflow is a system in which LLMs and tools are orchestrated through predefined code paths; an agent is a system in which LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasksSourced [Source 1].
OpenAI's practical guide defines an agent as a system that independently accomplishes tasks on your behalf, and says that applications that integrate LLMs but do not use them to control workflow execution (simple chatbots, single-turn LLM calls, sentiment classifiers) are not agents. As traits it lists (1) that an LLM manages workflow execution and makes decisions, recognizes when the workflow is complete, can correct its own actions if needed, and on failure can halt and hand control back to the user, and (2) that it has access to tools for interacting with external systems, selects among them dynamically depending on the state of the workflow, and always operates within clearly defined guardrailsSourced [Source 3].
ISO/IEC 22989:2022 (AI concepts and terminology) is said to define terms such as "AI agent" (a summary on the ISO page; the text is paid and was not checked) [Source 33]. The expressions differ, but what the two companies' descriptions share is that the LLM decides the flow of control.
For applications that use LLMs, Anthropic recommends "finding the simplest solution possible, and only increasing complexity when needed", which, it says, may mean not building agentic systems at all. It also says that agentic systems often trade latency and cost for better task performance. For many applications, it says, optimizing a single LLM call with retrieval and in-context examples is enoughSourced [Source 1].
3. The basic structure — LLM + tools + a loop
(1) The function-calling (tool use) round trip
The basics of tool use (function calling) are similar across developers. The Gemini API documentation shows this flow: (1) declare a function's name, parameters and purpose and pass it to the model; (2) instead of executing the function, the model returns the name of the function to call and its arguments; (3) executing the function is the calling application's responsibility, and the model does not execute it; (4) the result is sent back to the model, which produces a user-friendly answer. This can repeat over multiple turns, and several functions can be called at once (parallel) or in sequence (compositional)Sourced [Source 4]. The Claude documentation is similar: client tools run in the calling application; the model responds with stop_reason "tool_use" and the application sends back the result as a tool_result. Server tools such as search, on the other hand, run on the provider's infrastructureSourced [Source 5].
View original SVG, values and source annotations
This structure means the model only outputs "strings" for a function name and arguments, and real actions happen within the range the application allows. The heart of safety is what step 3 is allowed to do (Section 7). OpenAI's guide divides tools into three kinds: data (retrieve information: query a database, read a PDF, search the web), action (act on systems: update a record, send a message) and orchestration (an agent itself serves as a tool for another agent)Sourced [Source 3]. In research, Toolformer showed that a language model can learn from a handful of examples, in a self-supervised way, when and with which arguments to call APIs such as a calculator, question answering, search engines, translation and a calendar [Source 7].
(2) Interleaving reasoning and acting: ReAct
ReAct (Yao et al.) has an LLM generate reasoning traces (thoughts) and actions in an interleaved manner. Thoughts help create, track and update action plans and handle exceptions; actions let the model reach knowledge bases or environments for more information. The paper says that on question answering (HotpotQA) and fact verification (Fever), using a simple Wikipedia API, ReAct overcame the hallucination and error propagation seen in chain-of-thought reasoning, and on interactive decision-making benchmarks (ALFWorld and WebShop) it outperformed imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively (with only one or two in-context examples; models of the paper's time)Sourced [Source 6].
(3) The loop and stop conditions
Anthropic describes an agent as typically "just LLMs using tools based on environmental feedback in a loop". At each step it is crucial to gain "ground truth" from the environment, such as tool results or code execution, to assess progress; a task often ends on completion, but it is also common to include stopping conditions such as a maximum number of iterations to maintain control [Source 1]. OpenAI's guide calls the loop a "run" and gives as exit conditions the invocation of a final-output tool, a particular structured output, errors, reaching a maximum number of turns, and a response without tool calls [Source 3]Sourced.
4. Planning and memory
(1) Planning
In work that spans several steps, how the next step is chosen decides performance. ReAct updates its action plan by inserting thoughts. Tree of Thoughts (Yao et al.) generalizes chain of thought: it explores several paths over units of text ("thoughts"), self-evaluates choices to decide the next action, and looks ahead or backtracks where needed. The paper reports that on Game of 24, where GPT-4 with chain-of-thought prompting solved only 4% of tasks, it reached 74% (under the models and task conditions of the paper)Sourced [Source 11].
On the other hand, Valmeekam et al. report that on a benchmark based on domains of the International Planning Competition, the ability of LLMs to autonomously generate executable plans is "quite meager", averaging only about a 3% success rate. The heuristic and human-in-the-loop modes showed slightly more promiseSourced [Source 12]. These are results for models of early 2023, not for the reasoning models that came later (see this series' explainer on reasoning models). Planning ability can change a great deal between model generations (this article's commentary).
(2) Memory
An agent's memory can be sorted broadly into "short-term" and "long-term" (this article's grouping).
| Kind | What is stored | Examples (sources) | Cautions |
|---|---|---|---|
| Short-term (context) | The conversation so far and the tool calls and results, held in the model's context window | Returning results to the model in the function-calling exchange [Source 4] | The more steps, the longer the context, and the more has to be re-read each time (the calculation in Section 7) |
| Long-term (external memory) | Experience and knowledge stored outside the model, in natural language for example, and retrieved when needed | Generative Agents: a complete record of experiences in natural language, synthesized over time into higher-level "reflections" and retrieved dynamically to plan behavior (a sandbox of 25 agents) [Source 9] | The design of what to keep and what to retrieve decides quality |
| Hierarchical memory management | Moving data between fast and slow memory to give the appearance of a large memory within a limited context window | MemGPT: "virtual context management" inspired by the hierarchical memory of operating systems; evaluated on document analysis and multi-session chat [Source 10] | The paper evaluates two domains. Generalization to other uses was not checked |
| Recording lessons from failure | Verbal "reflections" on the outcome of a trial, kept in an episodic memory buffer | Reflexion: reinforces through linguistic feedback rather than weight updates. 91% pass@1 on HumanEval (the paper gives the GPT-4 of the time as 80%) [Source 8] | A value for the models of the paper's time, not a comparison with current models |
All Sourced (descriptions in each paper or document). The short-term/long-term grouping is this article's own.
5. Workflow patterns, and multiple agents
Before building an agent, first consider whether a workflow is enough. Anthropic takes as its basic building block the "augmented LLM" (an LLM enhanced with retrieval, tools and memory) and shows five workflow patterns built on itSourced [Source 1].
| Pattern | What it does (Anthropic's description) | Where it fits (Anthropic's description) |
|---|---|---|
| Prompt chaining | Decomposes a task into a sequence of fixed subtasks, each LLM call processing the output of the previous one. Programmatic checks (gates) can be placed in between | When the task can be cleanly decomposed into fixed subtasks; trades latency for accuracy by making each call easier |
| Routing | Classifies an input and directs it to a specialized follow-up task | When there are distinct categories better handled separately and classification can be done accurately |
| Parallelization | Runs subtasks in parallel (sectioning) or runs the same task several times and aggregates the results (voting) | When subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher confidence |
| Orchestrator-workers | A central LLM dynamically breaks a task down, delegates to worker LLMs and synthesizes their results | Complex tasks whose subtasks cannot be predicted (for example, coding, where the number of files to change depends on the task) |
| Evaluator-optimizer | One LLM generates a response while another gives evaluation and feedback in a loop | When there are clear evaluation criteria and iterative refinement gives measurable value |
All Sourced (Anthropic, dated 19 December 2024). The company says these are not prescriptive but common patterns to combine and tune.
When are multiple agents effective?
Anthropic's case study of its research feature (dated 13 June 2025) describes a lead agent that hands work to parallel subagents. It says that on the company's internal research evaluation a multi-agent system with Claude Opus 4 as lead and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2%, and that it is especially strong on breadth-first queries that pursue several independent directions at once. At the same time it says that "agents typically use about 4 times more tokens than chat interactions, and multi-agent systems about 15 times more", that economic viability requires tasks whose value exceeds the cost of the extra performance, and that domains in which all agents share the same context or have many dependencies (for example most coding tasks) are not a good fit today. In the company's analysis, token usage alone explained 80% of the variance in BrowseComp performanceSourced [Source 2]. This is the developer's internal evaluation and analysis; independent reproduction was not checked.
Cemri et al. collected more than 1,600 annotated traces (MAST-Data) from 7 popular multi-agent frameworks and built a failure taxonomy (MAST). Expert annotation identified 14 failure modes grouped into three categories: (i) system design issues, (ii) inter-agent misalignment and (iii) task verification. The study starts from the observation that the performance gains of multi-agent frameworks on popular benchmarks are often minimalSourced [Source 21].
6. What is actually being automated
Benchmark numbers change greatly with model generation and settings. The table below gives values shown by each paper or page, all results for the models at the time of publication. They are not the performance of current models and are not to be lined up to rank them.
| Benchmark (source) | What it measures | Reported value (at publication) | Cautions when reading |
|---|---|---|---|
| GAIA (Mialon et al., 2023) [Source 14] | 466 real-world questions that are conceptually simple but need reasoning, multimodality, web browsing and tool use | Human respondents 92%, GPT-4 with plugins 15% | Answers to 300 of the questions are withheld and used for the leaderboard |
| WebArena (Zhou et al., 2023) [Source 15] | Long tasks on working websites from four domains (e-commerce, forums, collaborative development, content management), measuring functional correctness | The best GPT-4-based agent: 14.41% task success; humans 78.24% | The best agent at the time of the paper; later results differ |
| AgentBench (Liu et al., 2023) [Source 16] | Reasoning and decision-making of LLMs as agents in 8 environments | Top commercial models are strong, with a large gap to many open models of 70B or less. The main causes of failure are weak long-term reasoning, decision-making and instruction following, the paper analyses | The paper says the effect of training on code differs by task |
| τ-bench (Yao et al., 2024) [Source 13] | Whether an agent follows domain API tools and policy in conversation with a (LLM-simulated) user; the final database state is compared with the goal state | Function-calling agents such as gpt-4o succeed on fewer than 50% of tasks; pass^8 below 25% in retail | pass^k is a reliability metric: succeeding every time in k repeats |
| TheAgentCompany (Xu et al., 2024) [Source 17] | Professional tasks in an environment imitating a small software company: browsing, writing code, running programs, communicating with coworkers | The most competitive agent autonomously completed 30% of tasks | The paper summarizes that a good portion of simpler tasks could be solved while harder long-horizon tasks are out of reach |
| Remote Labor Index (Mazeika et al., 2025) [Source 18] | End-to-end real freelance projects (240). Automation rate = share of projects whose deliverable is judged at least as good as the human standard | Even the highest-performing agent: 2.5% automation rate | Evaluation is done manually by trained experts. A value at the time of the paper; the leaderboard can be updated |
All Sourced (abstracts and pages of each paper). Numbers are values at publication.
There is also a way to measure by the "length" of tasks. METR (Kwa et al.) proposed the metric of the 50%-task-completion time horizon: the time humans typically take on tasks that AI models can complete with 50% success. The paper reports about 50 minutes for the frontier model of the time (Claude 3.7 Sonnet), says that this value for frontier models has been doubling approximately every seven months since 2019 (possibly accelerating in 2024), and extrapolates that if the results generalize to real software tasks, within 5 years AI systems could automate many software tasks that currently take humans a monthSourced [Source 19]. However, METR's own page (last updated 8 September 2026 and stated to be no longer actively updated) warns against these misreadings: (1) the 50% horizon is not how long an agent can act autonomously but a measure of task difficulty; (2) the tasks are self-contained and well specified, "cleaner" than real work, which draws on prior context and tacit knowledge; (3) the tasks are mainly software engineering, machine learning and cybersecurity, and values in other domains can differ greatly; (4) human durations are based on people with little context (a new hire, an outside contractor)Sourced [Source 20].
RLI's 2.5% and TheAgentCompany's 30% differ in the kind of task, the way of judging and the models of the time. Comparing the size of the numbers does not tell you "which is further along". Decide by (1) whether it resembles your own task, (2) how many trials the result comes from, (3) who scored it and how, and (4) whether a failure can be tolerated (this article's commentary).
7. Types of failure (with our calculation)
Agent failures are easier to counter when sorted into types. The following sorts failures and countermeasures that appear in primary sources (the grouping into types is this article's own).
(1) Compounding errors, and variation when repeated
Anthropic says that an agent's autonomy means higher costs and "the potential for compounding errors", and recommends extensive testing in sandboxed environments along with guardrails [Source 1]. Its research-feature case study likewise says that "agents are stateful and errors compound", and that because one cannot simply restart from the beginning when errors occur (restarts are expensive and frustrating), it built systems that can resume from where an error occurred [Source 2]Sourced.
Our calculation A simple model turns intuition into numbers.
- Chained steps: if each step succeeds independently with probability r, the probability that all n steps succeed is rn. For r = 0.99 and n = 50 it is about 60.5%; for r = 0.95 and n = 20 about 35.8%; for r = 0.9 and n = 10 about 34.9%. Even r = 0.999 over n = 100 gives about 90.5%
- Reliability over repeats: if a single trial succeeds with probability p, independently, the probability of succeeding in all k trials (an idealization of pass^k) is pk. For p = 0.9 and k = 8 it is about 43%; for p = 0.6 and k = 8 about 1.7%. Meanwhile the probability of "at least one success in k trials" is 1 - (1 - p)k, about 99.9% for p = 0.6 and k = 8. "Succeeds at least once" and "succeeds every time" are entirely different levels of performance
- Accumulating context: assume 3,000 tokens of initial context and 1,000 tokens added per step, and that step n reads 3,000 + (n - 1) x 1,000 tokens. The cumulative total is 3,000n + 1,000 x n(n - 1)/2: 525,000 tokens at n = 30 and 1,375,000 at n = 50. If the context did not accumulate (4,000 each time), it would be 200,000 at n = 50
Assumptions and limits: independence is an idealization; in practice earlier errors can cause later ones, and the agent may self-correct partway. τ-bench's pass^k is computed per task and averaged; the pk above is a formula for intuition. The context calculation is a simple model that ignores compression by summarization, caching and trimming of tool results, and is not a measurement of any real agent.
(2) Runaway behavior and cost
Without a stop condition in the loop, an agent can repeat the same operation or keep searching. Anthropic says that early agents in its research feature made errors such as spawning 50 subagents for simple queries, scouring the web endlessly for nonexistent sources, and distracting each other with excessive updates [Source 2]. The basic countermeasures are limits on the maximum number of iterations or token budget, and a mechanism that returns control to a person once a failure threshold is exceeded (OpenAI's guide names exceeding failure thresholds and high-risk actions as the two main triggers for human intervention) [Source 3]Sourced.
(3) Tool misuse, and instructions mixed in through external data
AgentBench names weak long-term reasoning, decision-making and instruction following as the main causes of failure [Source 16]. Moreover, because an agent works by reading external data (email, files, web pages), there is a danger that it follows instructions hidden in them. Greshake et al. pointed out that LLM-integrated applications "blur the line between data and instructions" and presented indirect prompt injection, which embeds instructions in data likely to be retrieved, together with a taxonomy of attacks. It includes demonstrations against real systems and states that effective mitigations are still lacking [Source 22]. NIST's CAISI explains agent hijacking as arising because "trusted internal instructions and untrusted external data are not clearly separated", and shows that evaluations should be adaptive rather than fixed and that measuring attack success over multiple attempts may give more realistic resultsSourced [Source 23]. This article does not cover concrete attack procedures.
(4) Permissions that are too large (excessive agency)
OWASP's "LLM06:2025 Excessive Agency" defines a vulnerability in which damaging actions are performed in response to unexpected, ambiguous or manipulated LLM output, and says the root cause is typically one or more of excessive functionality, excessive permissions and excessive autonomy. Its examples include an extension meant only to read that also includes functions to modify and delete, an extension meant to act in an individual user's context that uses a generic high-privileged identity with access to all users' files, and an extension that deletes documents without confirmation from the user (OWASP is an industry community project, not a law or a standard)Sourced [Source 24]. OpenAI's guide presents the idea of rating each tool low, medium or high in risk, considering read-only versus write access, reversibility, required account permissions and financial impact, and running guardrail checks or handing over to a person before high-risk functions execute [Source 3]Sourced.
(5) Coordination failures among multiple agents
The three MAST categories (system design, inter-agent misalignment, task verification) [Source 21] and the early errors Anthropic describes [Source 2] show that adding agents makes coordination complexity grow sharply. The cost issue is added: token usage can be about 15 times higher [Source 2]Sourced.
A standard operating procedure (SOP) for experiments or manufacturing is a "fixed path". It is close to what Anthropic calls a "workflow" (code decides the path), and in many situations predictability and consistency should matter more than an agent's flexibility [Source 1]. For example, a design worth considering first uses an LLM for individual steps such as importing measurement data, converting units, flagging outliers and drafting a report, with programmatic checks (gates) between the steps (this article's commentary). Control of the equipment itself should be left to deterministic control and interlocks, with the LLM limited to suggestions and drafts and a person approving execution (commentary; for equipment safety standards, see the ISO 10218 family covered in this series' explainer on world models and embodied AI).
OWASP's three root causes (excessive functionality, permissions and autonomy) apply directly to laboratories and factories. (1) Are the functions an agent can call the minimum necessary (are instrument start, stop and condition changes exposed?). (2) Are the account permissions it connects with enough for reading only (can it update or delete in the database?). (3) Is there human approval for irreversible operations (consuming a sample, overwriting instrument conditions, changing a lot record) [Sources 24, 3]. Unlike code and data, operations on physical samples and instruments sometimes cannot be undone. Starting with read-only tools and allowing writes bit by bit as a record builds up matches the idea in OpenAI's guide of keeping human oversight until confidence in reliability grows (commentary).
When an agent runs, record the input, tool call, output, model version and time of each step. Like a lab notebook, it is evidence for reproducing the same result later and tracing the cause of a failure. Because this field moves fast and behavior changes when the model version changes, pinning and recording versions matters (commentary). Evaluate not by a single success but by whether it succeeds every time when repeated (the idea of pass^k) and by overall success rate (yield). A multi-agent system can use about 15 times the tokens [Source 2], so an estimate of cost and electric power is also part of the adoption decision. Power is covered in this series' explainer on power and cooling for AI data centers.
8. Limits, and what could not be confirmed
(1) Performance of current models
The benchmark numbers in Sections 4 and 6 are values for the models at the time each paper or report was published. We have not covered results for each company's latest models as of 15 October 2026, or current leaderboard positions. METR's page is stated to have been last updated on 8 September 2026 and to be no longer actively updated.
(2) Developers' statements
Anthropic's multi-agent case (90.2%, about 4 and 15 times the tokens, 80% of the variance on BrowseComp) is the developer's internal evaluation and analysis; independent reproduction was not checked. OpenAI's function-calling announcement page refused automated retrieval, so the explanation of function calling rests on Google's and Anthropic's documentation.
(3) Standards
An international standard that directly defines the reliability or safety of agents could not be found. For ISO/IEC 22989 and the others, titles, editions and dates were checked within what the ISO pages show, and the texts were not checked (they are paid). NIST's AI Agent Standards Initiative (announced 17 February 2026) is at a stage where deliverables are yet to be published, and this article does not assert their content.
(4) What is not covered
The Model Context Protocol and agent interoperability are covered in the explainer on the Model Context Protocol, evaluation and safety of agents in more detail in the explainer on evaluating and securing agents, and code generation and screen operation in the explainer on coding agents and computer use. Feature or price comparisons of particular products are not covered. Concrete attack procedures are not written.
- An agent is "LLM + tools + a loop". The model does not execute functions; execution is the caller's responsibility. The heart of safety is what is permittedSourced
- A workflow (code decides the path) and an agent (the LLM decides) are different things. For many uses a simple structure is enough, and an agent trades latency and cost for performanceSourced
- The record of automation differs greatly by task, conditions and date of publication. Numbers at publication cannot be lined up to rank anythingSourced
- Types of failure: compounding errors (0.99 to the 50th power is about 61%), variation on repeats, runaway behavior and cost, instructions mixed in through external data, excessive permissions, and coordination failures among multiple agentsOur calculation / Sourced
- Standardization is still ahead. No standard directly addressing agents could be found, and NIST has only begun its workNot yet confirmed
9. How to read evaluations and standards
Evaluating agents has five sides: (1) task completion rate, (2) reliability when repeated, (3) the length and difficulty of tasks, (4) classification of the causes of failure, and (5) security. Unlike evaluating a model alone, the tools, environment, scoring method and number of trials all affect the result. Standards and guidance mostly give frameworks for AI risk management and terminology rather than for agents themselves, and standards that directly address agents are at an early stage of drafting.
| What is evaluated | Representative benchmarks and methods (name, source) | Standards and guidance (ISO/IEC, NIST and others) | Cautions when reading |
|---|---|---|---|
| Task completion (end to end) | GAIA (466 questions) [Source 14], WebArena (four domains of websites) [Source 15], AgentBench (8 environments) [Source 16], TheAgentCompany (an environment imitating a small company) [Source 17], Remote Labor Index (240 real freelance projects) [Source 18] | Not found | Numbers are for the models at publication. They change with the scorer (human or automatic), the kind of task and the number of trials. RLI's automation rate is a judgment of whether "a reasonable client would accept it", scored manually by experts Sourced |
| Reliability when repeated | τ-bench pass^k: the share of tasks solved in all k repeats (Yao et al.) [Source 13] | Not found | Measures instability that an average success rate hides. The paper's values are for models such as gpt-4o of the time Sourced |
| Length and difficulty of tasks | METR's 50%-task-completion time horizon (Kwa et al.) [Source 19]. The page (last updated 8 September 2026) [Source 20] | Not found | The horizon is not "the time an agent can act autonomously". Tasks are self-contained and centered on software, ML and cybersecurity; human duration is based on people with little context. It is the length of task at 50% success, not a length it completes every time Sourced |
| Classifying causes of failure | MAST: 14 failure modes of multi-agent systems in 3 categories, from more than 1,600 traces (Cemri et al.) [Source 21] | Not found | Targets multi-agent frameworks. Generalization to single agents was not checked beyond the paper Sourced |
| Security (hijacking and others) | Evaluation using AgentDojo (NIST CAISI technical blog, 17 January 2025; four environments) [Source 23]. Taxonomy of indirect prompt injection (Greshake et al.) [Source 22]. OWASP LLM06:2025 Excessive Agency [Source 24] | OWASP is an industry community document, not a law or a standard. NIST's RFI (January 2026; comments due 9 March) [Sources 25, 26] and the summary of responses, NIST 800-5 (18 May 2026) [Source 28], are not guidelines themselves | CAISI says evaluation is best adaptive and over multiple attempts. The design of permissions is central, not only component-by-component countermeasures Sourced |
| AI management, risk and terminology | (a framework, not a benchmark) | ISO/IEC 42001:2023 (AI management system) [Source 31]. ISO/IEC 23894:2023 (risk management) [Source 32]. ISO/IEC 22989:2022 (concepts and terminology; includes "AI agent") [Source 33]. NIST AI RMF 1.0 (January 2023) [Source 29] and AI 600-1 (July 2024; includes risks such as information security, human-AI configuration and value chain) [Source 30] | The two NIST documents are frameworks for voluntary use. ISO/IEC 42001 is a standard for an organization's management system and does not guarantee the safety of an individual agent. The texts are paid, so only titles and scope are given Sourced |
| Standardization work addressing agents directly | (standardization activity, not a benchmark) | NIST CAISI's AI Agent Standards Initiative (announced 17 February 2026; three pillars: industry-led standards, open-source protocols, and research on agent security and identity; research, guidelines and deliverables are said to be announced later) [Source 27] | Deliverables are still to come. Do not assume their content Not yet confirmed |
The organization of the table is this article's own (commentary). The texts of ISO/IEC standards are paid and were not checked. Checked 15 October 2026.
Three cautions for materials engineers
- Numbers change with settings and evaluator. Even for the same task, results change with how tools are given, the prompt, the scorer and the number of trials. A developer's internal evaluation is not an independent one. Do not read the value of a model at publication as the performance of a current model
- Look at "every time" and "the worst case" rather than the average. Measure reliability on repeats as pass^k does, and evaluate what can happen on failure (permissions, whether it can be undone) separately from the success rate. Saturation and contamination of benchmarks in general are covered in this series' explainer on benchmarks and how to read AI evaluations
- Tell "requirement", "recommendation" and "being drafted" apart in standards and guidance. NIST's frameworks and OWASP's document are not binding, and ISO/IEC 42001 sets requirements for an organization's management system without guaranteeing an agent's output. Standards that address agents directly are at an early stage, and legally binding regulation is covered in this series' explainer on the EU AI Act and others
10. Glossary
- AI agent
- A mechanism in which an LLM uses tools and proceeds in a loop while looking at results from the environment.
- Workflow
- A system that combines LLMs and tools through a predefined code path.
- Function calling (tool use)
- A mechanism in which the model returns the name and arguments of a function to call, and the application executes it and returns the result.
- ReAct
- A method that has the model generate reasoning traces and actions in an interleaved way.
- Ground truth (from the environment)
- The basis for judging progress, such as tool results or code-execution results.
- Stop condition
- A condition that ends the loop, such as a maximum number of iterations or a token budget.
- Orchestrator-workers
- A pattern in which a central LLM breaks a task down and delegates to worker LLMs.
- pass^k
- The share of tasks solved in all k repeats; a reliability metric.
- 50%-task-completion time horizon
- The human time needed for tasks that AI completes with 50% success.
- Indirect prompt injection
- Instructions hidden in external data an agent reads, inducing unintended actions.
- Agent hijacking
- A kind of indirect prompt injection; the term NIST CAISI uses.
- Excessive agency
- OWASP's term: functionality, permissions or autonomy larger than necessary.
- Guardrail
- A mechanism that limits an agent's actions, such as checks on input and output or risk ratings of tools.
- Memory (short-term and long-term)
- Memory within the context, and memory stored outside the model (this article's grouping).
11. References (primary sources)
- Anthropic "Building effective agents" (dated 19 December 2024; the page notes that much of the tooling landscape has changed since then). Checked 15 October 2026 — anthropic.com/engineering/building-effective-agents
- Anthropic "How we built our multi-agent research system" (dated 13 June 2025). Checked 15 October 2026 — anthropic.com/engineering/multi-agent-research-system
- OpenAI "A practical guide to building agents" (PDF; creation date in the file metadata is 7 April 2025). Checked 15 October 2026 — cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf
- Google "Function calling with the Gemini API" (page states last updated 23 September 2026). Checked 15 October 2026 — ai.google.dev/gemini-api/docs/function-calling
- Anthropic "Tool use with Claude" (Claude Platform Docs). Checked 15 October 2026 — platform.claude.com/docs/en/agents-and-tools/tool-use/overview
- Yao S. et al. "ReAct: Synergizing Reasoning and Acting in Language Models", arXiv:2210.03629 (2022) — arxiv.org/abs/2210.03629
- Schick T. et al. "Toolformer: Language Models Can Teach Themselves to Use Tools", arXiv:2302.04761 (2023) — arxiv.org/abs/2302.04761
- Shinn N. et al. "Reflexion: Language Agents with Verbal Reinforcement Learning", arXiv:2303.11366 (2023) — arxiv.org/abs/2303.11366
- Park J. S. et al. "Generative Agents: Interactive Simulacra of Human Behavior", arXiv:2304.03442 (2023) — arxiv.org/abs/2304.03442
- Packer C. et al. "MemGPT: Towards LLMs as Operating Systems", arXiv:2310.08560 (2023) — arxiv.org/abs/2310.08560
- Yao S. et al. "Tree of Thoughts: Deliberate Problem Solving with Large Language Models", arXiv:2305.10601 (2023) — arxiv.org/abs/2305.10601
- Valmeekam K. et al. "On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)", arXiv:2302.06706 (2023) — arxiv.org/abs/2302.06706
- Yao S. et al. "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains", arXiv:2406.12045 (2024) — arxiv.org/abs/2406.12045
- Mialon G. et al. "GAIA: a benchmark for General AI Assistants", arXiv:2311.12983 (2023) — arxiv.org/abs/2311.12983
- Zhou S. et al. "WebArena: A Realistic Web Environment for Building Autonomous Agents", arXiv:2307.13854 (2023) — arxiv.org/abs/2307.13854
- Liu X. et al. "AgentBench: Evaluating LLMs as Agents", arXiv:2308.03688 (2023) — arxiv.org/abs/2308.03688
- Xu F. F. et al. "TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks", arXiv:2412.14161 (2024) — arxiv.org/abs/2412.14161
- Mazeika M. et al. "Remote Labor Index: Measuring AI Automation of Remote Work", arXiv:2510.26787 (2025) — arxiv.org/abs/2510.26787
- Kwa T. et al. (METR) "Measuring AI Ability to Complete Long Software Tasks", NeurIPS 2025, arXiv:2503.14499 — arxiv.org/abs/2503.14499
- METR "Task-Completion Time Horizons of Frontier AI Models" (page last updated 8 September 2026 and stated to be no longer actively updated). Checked 15 October 2026 — metr.org/time-horizons
- Cemri M. et al. "Why Do Multi-Agent LLM Systems Fail?", arXiv:2503.13657 (2025) — arxiv.org/abs/2503.13657
- Greshake K. et al. "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection", arXiv:2302.12173 (2023) — arxiv.org/abs/2302.12173
- NIST CAISI "Technical Blog: Strengthening AI Agent Hijacking Evaluations" (dated 17 January 2025) — nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations
- OWASP Gen AI Security Project "LLM06:2025 Excessive Agency" (a document of an industry community project; not a law or a standard). Checked 15 October 2026 — genai.owasp.org/llmrisk/llm062025-excessive-agency
- NIST CAISI "CAISI Issues Request for Information About Securing AI Agent Systems" (dated 12 January 2026); the Federal Register notice is dated 8 January 2026, with comments due 9 March 2026 — nist.gov/news-events/news/2026/01/caisi-issues-request-information-about-securing-ai-agent-systems
- Federal Register "Request for Information Regarding Security Considerations for Artificial Intelligence Agents" (NIST, notice dated 8 January 2026) — federalregister.gov/documents/full_text/xml/2026/01/08/2026-00206.xml
- NIST CAISI "Announcing the AI Agent Standards Initiative for Interoperable and Secure Innovation" (dated 17 February 2026) — nist.gov/news-events/news/2026/02/announcing-ai-agent-standards-initiative-interoperable-and-secure
- NIST "Summary Analysis of Responses to the Request for Information Regarding Security Considerations for AI Agents" (NIST Trustworthy and Responsible AI 800-5, published 18 May 2026) — nist.gov/publications/summary-analysis-responses-request-information-regarding-security-considerations-ai
- NIST NIST AI 100-1, "Artificial Intelligence Risk Management Framework (AI RMF 1.0)" (January 2023) — doi.org/10.6028/NIST.AI.100-1
- NIST NIST AI 600-1, "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile" (July 2024) — doi.org/10.6028/NIST.AI.600-1
- ISO/IEC ISO/IEC 42001:2023, "Information technology — Artificial intelligence — Management system" (Edition 1, December 2023) — iso.org/standard/81230.html
- ISO/IEC ISO/IEC 23894:2023, "Information technology — Artificial intelligence — Guidance on risk management" (Edition 1, February 2023) — iso.org/standard/77304.html
- ISO/IEC ISO/IEC 22989:2022, "Information technology — Artificial intelligence — Artificial intelligence concepts and terminology" (Edition 1, July 2022; the ISO page summary notes it defines terms such as "AI agent") — iso.org/standard/74296.html
12. Claim-to-source audit
| Claim in the text | Basis | Class |
|---|---|---|
| Anthropic's distinction: workflows (LLMs and tools orchestrated through predefined code paths) versus agents (LLMs direct their own process and tool use); "the simplest solution possible"; the loop, ground truth from the environment and stop conditions; five workflow patterns (chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer); the note that much of the tooling has changed since December 2024 | Reference 1 https://www.anthropic.com/engineering/building-effective-agents | Sourced |
| OpenAI guide: definition and traits of an agent, what is not an agent, three tool types (data, action, orchestration), the "run" loop and exit conditions, tool risk ratings (read-only versus write, reversibility, permissions, financial impact), triggers for human intervention (exceeding failure thresholds, high-risk actions) | Reference 3 https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf | Sourced |
| The function-calling flow (declare, model returns name and arguments, the app executes, return the result; parallel and compositional calls; execution is the caller's responsibility); client tools versus server tools; stop_reason of tool_use | Reference 4 https://ai.google.dev/gemini-api/docs/function-calling Reference 5 https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview | Sourced |
| Toolformer (calculator, Q&A, search, translation, calendar); ReAct (+34% on ALFWorld, +10% on WebShop); Tree of Thoughts (74% on Game of 24 versus 4% for chain of thought); Valmeekam et al. (about 3% average success in autonomous plan generation) | Reference 7 https://arxiv.org/abs/2302.04761 Reference 6 https://arxiv.org/abs/2210.03629 Reference 11 https://arxiv.org/abs/2305.10601 Reference 12 https://arxiv.org/abs/2302.06706 | Sourced |
| Generative Agents (natural-language records, reflection, retrieval); MemGPT (virtual context management); Reflexion (verbal reflections stored in memory; 91% on HumanEval) | Reference 9 https://arxiv.org/abs/2304.03442 Reference 10 https://arxiv.org/abs/2310.08560 Reference 8 https://arxiv.org/abs/2303.11366 | Sourced |
| Multi-agent: beats a single agent by 90.2% on an internal eval; about 4x and 15x tokens; token usage explains 80% of variance on BrowseComp; poor fit for shared-context or dependency-heavy domains; early errors (50 subagents and so on); errors compound. MAST (1,600+ traces, 7 frameworks, 14 modes, 3 categories) | Reference 2 https://www.anthropic.com/engineering/multi-agent-research-system Reference 21 https://arxiv.org/abs/2503.13657 | Sourced |
| GAIA (466 questions; humans 92%, GPT-4 with plugins 15%); WebArena (14.41% versus 78.24%); AgentBench (8 environments); τ-bench (below 50%; pass^8 below 25%); TheAgentCompany (30%); Remote Labor Index (240 projects; 2.5%; the definition of automation rate). All for models at the time of publication | Reference 14 https://arxiv.org/abs/2311.12983 Reference 15 https://arxiv.org/abs/2307.13854 Reference 16 https://arxiv.org/abs/2308.03688 Reference 13 https://arxiv.org/abs/2406.12045 Reference 17 https://arxiv.org/abs/2412.14161 Reference 18 https://arxiv.org/abs/2510.26787 | Sourced |
| METR's 50% task-completion time horizon (about 50 minutes for Claude 3.7 Sonnet; doubling roughly every seven months since 2019; the extrapolation); the METR page (last updated 8 September 2026; not the time an agent can act autonomously; nature of tasks; baseline of human durations) | Reference 19 https://arxiv.org/abs/2503.14499 Reference 20 https://metr.org/time-horizons/ | Sourced |
| Indirect prompt injection (the blurred line between data and instructions); the NIST CAISI technical blog (17 January 2025; lack of separation between trusted internal instructions and external data; adaptive evaluation; multiple attempts; four AgentDojo environments); OWASP LLM06:2025 (excessive functionality, permissions and autonomy; examples) | Reference 22 https://arxiv.org/abs/2302.12173 Reference 23 https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations Reference 24 https://genai.owasp.org/llmrisk/llm062025-excessive-agency/ | Sourced |
| The NIST CAISI RFI (12 January 2026; Federal Register 8 January 2026; comments due 9 March 2026); the AI Agent Standards Initiative (17 February 2026; three pillars; deliverables to come); NIST 800-5 (published 18 May 2026; summary of RFI responses) | Reference 25 https://www.nist.gov/news-events/news/2026/01/caisi-issues-request-information-about-securing-ai-agent-systems Reference 26 https://www.federalregister.gov/documents/full_text/xml/2026/01/08/2026-00206.xml Reference 27 https://www.nist.gov/news-events/news/2026/02/announcing-ai-agent-standards-initiative-interoperable-and-secure Reference 28 https://www.nist.gov/publications/summary-analysis-responses-request-information-regarding-security-considerations-ai | Sourced |
| ISO/IEC 42001:2023, 23894:2023, 22989:2022 (the ISO page summary notes it includes "AI agent"); NIST AI 100-1 (January 2023); NIST AI 600-1 (July 2024). Numbers, titles and editions follow official pages; ISO full texts were not checked | Reference 31 https://www.iso.org/standard/81230.html Reference 32 https://www.iso.org/standard/77304.html Reference 33 https://www.iso.org/standard/74296.html Reference 29 https://doi.org/10.6028/NIST.AI.100-1 Reference 30 https://doi.org/10.6028/NIST.AI.600-1 | Sourced |
| Success over chained steps r^n (0.99^50 = about 60.5%; 0.95^20 = about 35.8%; 0.9^10 = about 34.9%; 0.999^100 = about 90.5%); repeated success p^k (0.9^8 = about 43%; 0.6^8 = about 1.7%; at least one success about 99.9%); cumulative context (525,000 at n = 30; 1,375,000 at n = 50; 200,000 without accumulation) | Our calculation under an independence idealization and an assumption of 3,000 tokens of context plus 1,000 per step. Not measurements of any real agent. τ-bench's pass^k is computed per task and averaged; p^k is an intuition-building formula | Our calculation |
| An international standard that directly defines the reliability and safety of agents; independent reproduction of developers' evaluations; results for the latest models; deliverables of the NIST AI Agent Standards Initiative; the full texts of ISO/IEC standards; the full text of OpenAI's function-calling announcement page | Could not be confirmed in the primary sources accessible within this article's scope, so not stated | Commentary |
| The readings "start with a workflow when a procedure exists", "start read-only and put writes and irreversible actions under human approval", "treat run logs as lab notebooks and evaluate by repetition like yield", the tables in Sections 4 and 9, and the short-term/long-term grouping of memory | This article's own organization and explanation based on published content; not views expressed by any company or institution | Commentary |
| Figs. 2 to 5 are explanatory drawings, not measured data; Figs. 1 and 6 are ImageGen-generated conceptual illustrations | A note by this article | Commentary |
Last updated 15 October 2026. Sources are limited to primary material (developers' technical documents, arXiv papers, official pages of METR, NIST and OWASP, and ISO/IEC standard pages). The texts of ISO/IEC standards are paid and could not be checked. Because the article includes readings and organization of its own, these are marked as Commentary and kept separate from sourced fact. Results for the latest models, independent reproduction of developers' evaluations, a standard that directly defines agents, deliverables of NIST's AI Agent Standards Initiative, and the texts of the standards could not be confirmed and are not stated here. Benchmark numbers are for the time of publication and the settings given, and do not show which model is better. Concrete attack procedures are not given. Figs. 2 to 5 are vector drawings; Figs. 1 and 6 are ImageGen-generated conceptual illustrations (none of them shows a real measurement result or a physical product).