AI Agents for Materials Research Explained

TECHNOLOGY EXPLAINER

AI Agents for Materials Research
— when an LLM uses calculations, databases and lab equipment as "tools"

An LLM can use literature search, databases, calculation software and even lab equipment as tools to carry out materials look-ups and experiments. That is an AI agent for materials research. The general mechanism of agents (LLM, tools and a loop) is left to the series explainer on how AI agents work; this article concentrates on what is specific to materials research. Which tools can be used? What did the papers actually confirm in real cases (ChemCrow, Coscientist, LLaMP, MDCrow, CRESt and others)? Where does it go wrong when measured? And what should a materials engineer check before adopting one? It is all set out within what the papers state, without exaggeration.

Built from the papers on ChemCrow (Bran et al.), Coscientist (Boiko et al.), LLaMP (Chiang et al.), MDCrow (Campbell et al.), CRESt (Zhang et al.), AFMBench (Mandal et al.) and MatTools; the MCP specification; and ISO/IEC and NIST pages. Worked numbers are our own calculations. Checked 22 October 2026

Fig. 1. Conceptual illustration generated with ImageGen. Reading literature, running calculations and conducting experiments are separate roles. Connecting tools does not remove the need to check inputs and results.Not an actual product, facility or measurement. Enlarged models, lines, arrows and colours are explanatory.

What this article covers

The short version / what to know first / place in the series
Splitting materials work into "tools"
How tools are connected: functions, code, manuals and MCP
Case 1: ChemCrow, down to a synthesis robot
Case 2: Coscientist, liquid handling and reaction optimization
Case 3: computation, data and discovery robots (LLaMP, MDCrow, CRESt and others)
What measurement showed: where they stumbled
Design specific to materials work: gates and how much to delegate
A materials engineer's view
Limits, what could not be confirmed, and a summary
How to read the yardsticks and standards
Glossary
References
Claim-to-source audit

How claims are labelled in this article

Sourced = stated in published material (link given)
Our calculation = a figure this article derived, with the assumptions spelled out
Not yet confirmed = a plan, target, draft or other item that is not settled
Interpretations for materials engineers are marked separately as commentary; inference is shown in bold; claims by research groups or companies are labelled as claims.

How to read this article: it is not "AI discovered a material"

Most cases covered here are syntheses of known compounds, computational searches or optimization in a specific system. Calculated predictions and results confirmed by synthesis and measurement are written separately. Statements by research groups that they "discovered" or "improved" something are treated as claims, and where independent verification could not be confirmed, we say so.

1. The short version / what to know first / place in the series

A materials-research agent is an LLM that picks a tool, calls it and reads the result. The tools are search of papers and manuals, databases (such as the Materials Project), calculation software, code execution and lab equipmentSourced[Ref. 2, 3, 1]
In real cases the papers report agents operating a liquid-handling robot or a synthesis robot and running experiments (Coscientist, ChemCrow). But the targets are mostly known reactions and compounds, and some human handling remains (such as moving plates)Sourced[Ref. 2, 1]
Measured, they stumble. A model strong on materials question answering did poorly operating lab equipment (AFMBench), and an LLM used as the grader missed fluent mistakes (ChemCrow). The one thing to remember: you need a gate that checks the source, conditions and units of what a tool returned, and human approval before equipment movesSourced[Ref. 10, 1]

What to know before reading

An "agent" is a system in which an LLM repeatedly calls tools, reads the results and decides the next action (see the series explainers on how AI agents work and on Transformers and LLMs). The difference between workflows and autonomous agents, ReAct and the accumulation of errors are explained there and not repeated here. This article can be read alone; terms specific to materials are explained where they first appear.

Place in the series

Extracting data from papers is in the explainer on LLMs and text mining for materials; running experiments automatically and deciding the next one is in the explainers on self-driving laboratories and on closed-loop optimization; materials calculations are in those on quantum chemistry and materials simulation and on machine-learning interatomic potentials. This article is about the LLM as the coordinator that links those tools.

2. Splitting materials work into "tools"

Imagine asking a new colleague to "look up the elastic modulus of this composition and suggest three candidates". They would search the literature, query a database, run calculation software and, if needed, ask for an experiment. An agent is a system in which an LLM plays that new colleague. The LLM does not calculate by itself; it asks tools and reads what comes back.

Fig. 2. Conceptual illustration generated with ImageGen. 1 search literature, 2 calculate, 3 experiment. LLM assistance does not automatically validate calculations or experiments.Not an actual product, facility or measurement. Enlarged models, lines, arrows and colours are explanatory.

View original SVG, values and sources

Original SVG reference: Conceptual diagram (vector drawing).A grouping of the tools an LLM can use. Examples are from the ChemCrow, Coscientist, LLaMP, MDCrow, MatTools and AILA papers.

Where the analogy breaks

A new colleague senses that an odd value is wrong and can check with someone. An LLM may tidy up an odd value from a tool into a plausible sentence (commentary; the same kind of problem as the reported hallucinations). The ChemCrow authors start from the point that an LLM answering chemistry questions directly can be "hyperconfident yet typically wrong", which is why they give it toolsSourced[Ref. 1].

Each kind of tool has cautions specific to materials.

Search: papers are biased toward successes (see the series explainer on LLMs and text mining). Searching an instrument's manual is decisive when the LLM must write operating codeSourced[Ref. 2]
Databases: the Materials Project is a DOE effort that pre-computes material properties and publishes themSourced[Ref. 13, 14]. Calculated and measured values can be mixed, so check whether a returned value is calculated or measured. The LLaMP authors say errors seem to come from mixed data sourcesSourced[Ref. 3]
Calculation software: results depend on calculation conditions (functional, cutoff and so on); do not let calculated values be written up as measured ones (see the explainer on density functional theory)
Lab equipment: it does not stay inside the computer and affects the physical world; reagent handling and safety are involved (Chapter 8)

3. How tools are connected: functions, code, manuals and MCP

There are four main ways to connect an LLM to tools.

1 Function calls: call prepared functions (for example a database search) in a fixed format. ChemCrow has 18 expert-designed tools covering reaction, molecule, safety and searchSourced[Ref. 1]; MDCrow has 40 toolsSourced[Ref. 4]
2 Write and run code: the LLM writes Python and runs it in an isolated environment. The PYTHON command of Coscientist runs in an isolated containerSourced[Ref. 2]; MatTools tests how well an LLM writes code that uses pymatgenSourced[Ref. 11]
3 Read the manual, then write operating code: retrieve an instrument's manual and have the LLM read it. Coscientist read the manual of a liquid-handling-robot module released after GPT-4's training data cutoff and wrote the code to operate itSourced[Ref. 2]
4 A common connection protocol (MCP): to save building a connection for every tool there are open protocols linking LLM applications and tools. MCP (Model Context Protocol) has servers offer resources, prompts and tools, and the specification is based on JSON-RPC 2.0Sourced[Ref. 12]

On safety the MCP specification says tools represent arbitrary code execution and must be treated with caution, and that explicit user consent must be obtained before invoking a tool. It also says MCP itself cannot enforce these principles at the protocol levelSourced[Ref. 12] (revision checked: 2026-07-28). MCP is an open specification, not a standard from a standards body. Attempts to connect materials instruments this way have appeared as preprints and similar, but their content was not checked hereNot yet confirmed.

The next figure splits one tool-using exchange among the person, the LLM and the tool. The example was made up for this article.

Fig. 3 Figure including our calculation (vector drawing).One tool call and where checks belong. For sources such as the Materials Project, which hold many calculated values, the distinction between calculated and measured matters especially (commentary).

The calculation under the figure shows how large a unit slip can be. A formation energy of −0.52 eV/atom is about −50.2 kJ/mol. One eV per atom is the product of the elementary charge (1.602176634×10⁻¹⁹ C) and the Avogadro constant (6.02214076×10²³ /mol), about 96.485 kJ/molOur calculationSourced[Ref. 15, 16]. A factor of about 96 might be noticed from the order of magnitude, but without a gate where a person checks the units of what a tool returned it can pass unnoticed.

Easily confused: calling a function versus writing and running code

A function call is like pressing a fixed button: limited, but predictable. Writing code is like having it jot down on blank paper what it wants done: very flexible, but you must check what the code does. With the latter, narrow the scope of execution, for example by running it in an isolated environment. Where the analogy breaks: many real systems combine the two.

4. Case 1: ChemCrow, down to a synthesis robot

ChemCrow (Bran et al., 2024) is a chemistry agent that combines GPT-4 with 18 expert-designed tools for reactions, molecules, safety, search and so onSourced[Ref. 1]. The paper reports the following.

Planning and running syntheses: on IBM's cloud-connected synthesis robot, RoboRXN, it autonomously ran the synthesis of an insect repellent (DEET) and of three known thiourea organocatalysts, and all four gave the expected compoundsSourced[Ref. 1]. The paper adds that the tools could be used one by one with likely the same result; ChemCrow automates their execution through the reasoning of an LLM
Fixing the robot's complaints on its own: a predicted procedure is sometimes not directly executable on RoboRXN (for example not enough solvent). A person would normally fix it, but ChemCrow queried the platform's validation and revised the procedure until it was validSourced[Ref. 1]
Chromophore search (human-AI collaboration): ChemCrow was asked to train a machine-learning model (a random forest) to screen candidate chromophores and to suggest a molecule for a target absorption maximum of 369 nm. The proposed molecule was synthesized and measured and confirmed as a new chromophore, but its measured absorption maximum was 336 nm, 33 nm shorter than the targetSourcedOur calculation[Ref. 1]. The paper calls it "approximately the desired property"

A materials engineer's view 1: write the gap between prediction and experiment as it is

The chromophore case shows a view that matters on the shop floor. The machine-learning proposal (target 369 nm) and the measurement (336 nm) differed by 33 nm, about 9% of the targetOur calculation (369 − 336 = 33; 33 / 369 ≈ 0.089). That is not a "failure"; it is the intended use, narrowing with prediction and confirming by experiment (commentary). When reading an agent report, check that the gap between proposal and measurement is stated.

Easily confused: what happens when an LLM does the grading

The ChemCrow paper used both expert chemists (n = 4) and an LLM grader (EvaluatorGPT). The experts preferred ChemCrow for factual accuracy and completeness, but EvaluatorGPT preferred the answers of plain GPT-4, which looked fluent and complete. A grader without the knowledge to judge the facts cannot give a trustworthy assessment, the authors saySourced[Ref. 1]. Where factual correctness matters, always check LLM grading against human grading.

5. Case 2: Coscientist, liquid handling and reaction optimization

Coscientist (Boiko et al., Nature 2023) has GPT-4 act as a Planner that uses four commands: web search, Python execution, manual search and experimentSourced[Ref. 2]. The paper shows its abilities on six tasks. The published content is arranged in the figure.

Fig. 4 Conceptual diagram (vector drawing).Based on Boiko et al. (Nature 2023). The number of experiments, the definition of success and whether results were verified independently were not checked beyond what the paper states.

Experiments: on a liquid-handling robot (Opentrons OT-2) it designed and ran palladium-catalysed Suzuki and Sonogashira reactions. Analysis of the mixtures by GC-MS found signals of the target products in both cases, the paper saysSourced[Ref. 2]
The human part: plates were moved by hand, so it is not fully automated; yet the paper states that no human decision-making was involvedSourced[Ref. 2]. "Autonomous" and "some manual handling remains" can both be true, so check what the people actually did
Self-correction: it used a wrong method name, looked it up through the manual search, fixed the code and ran it successfullySourced[Ref. 2]
Synthesis-planning test: all non-browsing models got the synthesis of ibuprofen wrong, and the authors stress grounding the model in external information to avoid hallucinationSourced[Ref. 2]
Reaction optimization: compared with standard Bayesian optimization, two GPT-4-based approaches scored higher, the paper says, while also noting that the exploration-exploitation balance differs, so this may not indicate performance by itselfSourced[Ref. 2] (Bayesian optimization is in the series explainer on Bayesian optimization and active learning)
Safety: the authors note safety and dual-use (misuse) concerns for automated experimentation and include a brief study in the supplementSourced[Ref. 2]

Common misconception: "Coscientist and ChemCrow discovered new materials or reactions"

Strictly, no. What ChemCrow synthesized were the insect repellent DEET and three known organocatalysts, and Coscientist ran well-known coupling reactions. Both demonstrate that an agent can plan and execute known reactions itself using tools and equipment, not the discovery of a new reaction or material. The new chromophore of ChemCrow was a human-AI collaboration in which a machine-learning proposal was synthesized and checked.

6. Case 3: computation, data and discovery robots (LLaMP, MDCrow, CRESt and others)

Fig. 5 Conceptual diagram (vector drawing).Examples arranged by whether they connect to equipment and by publication status. Those with equipment cover known reactions or specific systems, and independent verification was not checked here.

(1) Examples that use calculations and databases

LLaMP (Chiang et al.): a hierarchical retrieval-augmented-generation (RAG) agent that queries the Materials Project and runs atomistic simulations. It works without fine-tuning and is reported to reduce errors in bulk modulus, band gap and formation energySourced[Ref. 3]. The reduction is as measured by the authors' own metric (self-consistency combining uncertainty and confidence)
MDCrow (Campbell et al.): an agent with 40 tools that automates molecular-dynamics workflows. Evaluated on 25 tasks, gpt-4o completed complex tasks with low variance, followed by the open model llama3-405b. Prompt style did not affect the best models but strongly affected smaller onesSourced[Ref. 4]
ChatMOF (Kang and Kim): a system for searching, predicting and generating MOFs, built from an agent, a toolkit and an evaluator. With GPT-4 it reports accuracies of 96.9% for searching, 95.7% for predicting and 87.5% for generating (on tasks the authors defined)Sourced[Ref. 5]

(2) A discovery example connected to robots

CRESt (Zhang et al., Nature 2025) is a platform that combines large multimodal models (handling compositions, text embeddings and microstructure images) with knowledge-assisted Bayesian optimization and robotic automation. Cameras watch the experiments, and a vision-language model proposes hypotheses to diagnose and correct anomalies. Applied to electrochemical formate oxidation, it explored more than 900 catalyst chemistries and 3,500 electrochemical tests in 3 months and found, in an eight-element space (Pd-Pt-Cu-Au-Ir-Ce-Nb-Cr), a catalyst with a 9.3-fold improvement in cost-specific performance, the authors saySourced[Ref. 6]. This is the authors' claim, and independent third-party verification could not be confirmed hereNot yet confirmed. The baseline and definition of "cost-specific performance" were checked only to the extent of the abstract.

(3) Cases treated as claims

SciAgents (Ghafarollahi and Buehler) generates hypotheses about bio-inspired materials using knowledge graphs and several agents, and the authors say it surpasses human-driven research in scale and precisionSourced[Ref. 7]. AtomAgents designs alloys with several agents[Ref. 8], and MatPilot goes from hypotheses to automated experiments through human-AI collaboration[Ref. 9]. These are presented as the authors' claims. The last two are preprints with no journal version found, and independent verification was not confirmedNot yet confirmed.

System	Tools and connections	Physical experiments	Verification described in the paper (as checked)
ChemCrow	18 tools; RoboRXN (a synthesis robot)	Yes (DEET, three known catalysts, one chromophore)	Outcome of the syntheses and the measured absorption maximum of the chromophore (336 nm)Sourced
Coscientist	Search, Python, manual search, a liquid-handling robot	Yes (Suzuki and Sonogashira); plate transfer by hand	Target-product signals confirmed by GC-MSSourced
LLaMP	Materials Project; atomistic simulation	No (computation and data)	Compared with plain LLMs by a self-consistency metricSourced
MDCrow	40 tools (MD set-up and analysis)	No (computation)	Completion across 25 tasksSourced
ChatMOF	Data retrieval, ML prediction, structure generation	No (computation)	Accuracy on tasks defined by the authorsSourced
CRESt	Multimodal models, Bayesian optimization, robots and cameras	Yes (over 900 compositions, 3,500 tests)	The authors' claim; independent verification not confirmedNot yet confirmed

7. What measurement showed: where they stumbled

Beyond the successes of the cases, look at what happened when they were measured. The published measurements are set side by side.

Fig. 6 Conceptual diagram (vector drawing).MatTools and AFMBench are from arXiv abstracts, ChemCrow and Coscientist from the published papers, and LLaMP from its arXiv abstract.

AFMBench (Mandal et al.): a framework that automates atomic force microscopy with LLM agents (AILA) and an evaluation suite for it. Even state-of-the-art models struggled with basic tasks and coordination, and Claude 3.5 Sonnet, strong on materials question answering, performed unexpectedly poorly as an agent. The authors say domain question-answering strength does not necessarily translate into agentic ability and conclude that rigorous benchmarking and prompt engineering are needed before deployment in labsSourced[Ref. 10]
MatTools: measures the ability to use materials tools through 69,225 questions about pymatgen and 49 real code-generation tasks (138 subtasks). The authors list three findings: generalists outshine specialists, AI knows AI, and simpler is betterSourced[Ref. 11].

Common misconception: "an LLM strong at materials questions will also operate lab equipment well"

This is not correct. The AFMBench authors report a model strong at materials question answering that did poorly as an agentSourced[Ref. 10]. A score on a knowledge test and the ability to use tools correctly, in order, and obey instructions are different abilities. The right check is to turn the tools and tasks you actually use into a test and measure before adoption.

8. Design specific to materials work: gates and how much to delegate

In materials work the values an agent returns become the basis for the next experiment or investment. So consider a design that places "gates" between the LLM's proposal and the equipment (our own arrangement, not a standard procedure).

Fig. 7 Conceptual diagram (vector drawing).An example of where gates and human approval can sit. The "gates" are our own summary (commentary).

Gate 1 (values and conditions): check the unit, data ID, calculated or measured, and calculation conditions of the returned value, and compare with known values. This is where a materials engineer's expertise counts most
Human approval: before equipment moves, a person checks the purpose, the substances and amounts, and safety, and the approval is recorded. The MCP specification likewise calls for explicit consent before invoking a toolSourced[Ref. 12]
Gate 2 (limits on the equipment side): AFMBench reports that an LLM can deviate from instructionsSourced[Ref. 10]. Restrict the amounts, temperatures and substances that can be operated outside the LLM, in equipment or software settings, rather than only asking the LLM to comply (commentary)
Records: keep the model name and version, the prompt, the tool calls and returned values, and who decided. Being able to trace why an experiment was run later is the basis of reproducibility

The other design axis is how much to delegate. The levels below set it out.

Fig. 8 Conceptual diagram (vector drawing).Before moving up a level, measure the system on your own tasks at that level (the idea behind AFMBench).

Easily confused: why draw a line between Level 2 and Level 3

Up to Level 2, a mistake can be undone by discarding the result inside the computer. From Level 3, reagents are used, equipment runs and effects that cannot be undone appear. The kind of risk changes even though it is the same "agent", so move to Level 3 only after gates, approval and equipment-side limits are in place (commentary).

9. A materials engineer's view

A materials engineer's view 2: first make your own equipment and software into "tools"

Before adopting an agent, ask whether your analytical instruments and calculation software can already be called as tools in a fixed way from outside. Decide the input format (including units), the output format, logging and how failures are returned. Just as Coscientist could read an instrument manual and write operating codeSourced[Ref. 2], the quality of the manual counts (commentary). This preparation helps automation and reproducibility even without an LLM.

A materials engineer's view 3: what is easy to delegate and what needs expert judgment

Easy to delegate: searching papers and manuals, preparing calculation input files, tidying results, running routine calculations. Needs expert judgment: what the goal is, whether calculation conditions suit the target, whether returned values are plausible, experimental safety, interpreting results, and designing confirmation experiments (controls, reproducibility, lot-to-lot variation). The real cases also work within a frame where people set the goal and check results (commentary).

Fig. 9. Conceptual illustration generated with ImageGen. People check the sample, conditions and records before the next action. Completing an operation is not the same as establishing a scientifically valid conclusion.Not an actual product, facility or measurement. Enlarged models, lines, arrows and colours are explanatory.

A materials engineer's view 4: measure on your own tasks before adoption

As with AFMBench, build a test from the tools and tasks you will actually use before adoptionSourced[Ref. 10]. For example: can it reproduce ten past calculations from the instructions alone? Does it notice inputs in the wrong unit? Record results with the model name and version and the prompt. Results change when the model or prompt changes (in MDCrow smaller models were strongly affected by prompt styleSourced[Ref. 4]), so measure again at every change.

10. Limits, what could not be confirmed, and a summary

(1) What could not be confirmed

The main text of CRESt (behind a subscription) was not read; only the abstract and Extended Data legends. The baseline of the 9.3-fold figure and independent verification could not be confirmed.
The number of experiments, detailed definitions of success and the existence of independent verification for Coscientist and ChemCrow were not checked beyond what the papers state.
Detailed figures for LLaMP, MDCrow, MatTools and AFMBench were checked at the level of abstracts (published versions may differ).
AtomAgents and MatPilot are preprints with no journal version found.
Preprints on controlling materials and chemistry equipment through MCP exist, but their content was not checked.
Repeat tests with the latest model versions and licences (including commercial use) were not checked.
An international standard specific to agents in materials research was not found. ISO/IEC standard texts are paid, so only the scope is described.

Summary

A materials-research agent is an LLM using search, databases, calculation software, code execution and lab equipment as tools. Each tool has its own cautions (calculated or measured, units, conditions)Sourced
The cases that went as far as experiments (Coscientist, ChemCrow) mostly targeted known reactions and compounds, and some human handling remains. They demonstrate planning and execution, not discoverySourced
Measured, there are cases of strength at materials questions yet weakness in the lab, and of LLM grading pulled by fluency. Measure on your own tasks before adoptionSourced
Build in gates for values and conditions, human approval, equipment-side limits and records, and draw a line between Level 2 (inside the computer) and Level 3 (operating equipment) (commentary)
Read figures such as the 9.3-fold of CRESt as claims, and check the baseline and independent verificationNot yet confirmed

11. How to read the yardsticks and standards

The first yardsticks for agents are whether they use tools correctly (tests of tool use such as MatTools) and whether they can complete lab or computational work (AFMBench, the 25 MDCrow tasks and so on). A major issue is then how the quality of answers is assessed: by experts or by an LLM grader. None of these measures whether a materials value returned by an agent is correct in an experiment. Checked on 22 October 2026.

What is assessed	Representative benchmark or method (name, source)	Standards and guidance (ISO/IEC, NIST, open specifications)	How to read it
Tool use	MatTools (69,225 pymatgen questions; 49 tasks and 138 subtasks)[Ref. 11]Sourced	Not found	A test of code writing; separate from performance on your own tools and tasks
Completing lab or computational work	AFMBench (AILA)[Ref. 10], the 25 MDCrow tasks[Ref. 4]Sourced	Not found	May not match question-answering scores; varies with the prompt
How answer quality is assessed	Experts (n = 4) together with an LLM grader (ChemCrow)[Ref. 1]Sourced	Not found (the ChemCrow paper says a standardized evaluation technique has not yet been established)	An LLM grader can be pulled by fluency; have people verify factual correctness
Confirmation by experiment	Target products confirmed by GC-MS (Coscientist); the measured absorption maximum of the chromophore (ChemCrow)[Ref. 2, 1]Sourced	Not found	Check, paper by paper, the number of experiments, the definition of success, the human part and whether verification was independent
Tool-connection protocol	Not found	The MCP specification (revision checked 2026-07-28), an open specification rather than a standard from a standards body[Ref. 12]Sourced	States principles on consent and tool safety, but the protocol itself cannot enforce them
AI risk and management	Not found	ISO/IEC 23894:2023 (guidance on AI risk management)[Ref. 18], ISO/IEC 42001:2023 (AI management system)[Ref. 19], NIST AI RMF 1.0 (NIST AI 100-1)[Ref. 20] and NIST AI 600-1 (Generative AI Profile)[Ref. 21]Sourced	23894 is guidance, 42001 sets requirements for organizations, and the NIST documents are voluntary. None guarantees the correctness of values an agent returns
Terminology	Not found	ISO/IEC 22989:2022 (AI concepts and terminology)[Ref. 17]Sourced	Helps unify terms; does not define the performance of materials agents

The arrangement of the table is Our calculation (organization). Numbers, titles and status checked on the ISO, NIST and MCP pages (22 October 2026). Standard texts are paid, so only the scope is described and requirements are not.

Reading notes (for materials engineers)

A benchmark ranking is not performance on your work: the results of MatTools and AFMBench hold for that paper's tasks, models and prompts. Measure again on your own tools and tasks
Separate "proposal" from "confirmed by experiment": an agent's output is a proposal. As with the ChemCrow chromophore, state the synthesized and measured result and its gap from the prediction
Separate requirement from recommendation: the NIST documents are voluntary, ISO/IEC 23894 is guidance, 42001 sets requirements for organizations and MCP is an open specification. Also separate plan from achievement (the CRESt result is a claim)

12. Glossary

AI agent: A system in which an LLM repeatedly calls tools, reads results and decides the next action.

Tool: A function an LLM calls, such as search, calculation or lab equipment.

Function calling: Calling prepared functions in a fixed format.

MCP (Model Context Protocol): An open protocol connecting LLM applications with tools and data.

API: An entry point for calling software or equipment from outside in a fixed way.

RAG (retrieval-augmented generation): Searching documents before answering and writing the answer from what was found.

Materials Project: A DOE effort that pre-computes material properties and publishes them as a database.

pymatgen: A Python library for handling crystal structures and other materials data.

Molecular dynamics: A simulation that follows atomic motion over time (see another article in this series).

Liquid-handling robot: A machine that dispenses fixed amounts of samples and reagents; for example the Opentrons OT-2.

Cloud lab: An automated experimental facility to which experiments can be submitted and run remotely.

GC-MS: Gas chromatography-mass spectrometry: separates a mixture and identifies its components.

Benchmark: A test measuring a model's ability on defined tasks.

LLM grader: Having another LLM grade answers; it can be pulled by fluency.

Hallucination: Writing plausible content that is unfounded or wrong.

Multi-agent: A set-up in which several agents with different roles cooperate.

Isolated environment (container): A closed area in which generated code runs so it cannot affect the user's machine.

Dual use: The property of technology that can serve both good and harmful ends.

Grounding: Tying a model's answers to reliable external information.

Bayesian optimization: A way of finding optimal conditions with few trials (see another article in this series).

Interlock: A safety mechanism that keeps equipment from running, or stops it, unless conditions are met.

ReAct: A way of using an LLM that alternates thinking, acting and observing results.

Prompt fragility: Results changing a lot with small changes in the instructions.

13. References

Bran A. M. et al. "Augmenting large language models with chemistry tools", Nat. Mach. Intell. 6, 525–535 (2024), DOI 10.1038/s42256-024-00832-8 — doi.org/10.1038/s42256-024-00832-8
Boiko D. A., MacKnight R., Kline B., Gomes G. "Autonomous chemical research with large language models", Nature 624, 570–578 (2023), DOI 10.1038/s41586-023-06792-0 — doi.org/10.1038/s41586-023-06792-0
Chiang Y. et al. "LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation", arXiv:2401.17244 (2024); published in Proc. EMNLP 2025, 25200–25232, DOI 10.18653/v1/2025.emnlp-main.1280 (statements here are from the arXiv abstract) — arxiv.org/abs/2401.17244
Campbell Q. et al. "MDCrow: automating molecular dynamics workflows with large language models", Mach. Learn.: Sci. Technol. 7, 025037 (2026), DOI 10.1088/2632-2153/ae4b07 (abstract checked in arXiv:2502.09565) — arxiv.org/abs/2502.09565
Kang Y., Kim J. "ChatMOF: an artificial intelligence system for predicting and generating metal-organic frameworks using large language models", Nat. Commun. 15, 4705 (2024), DOI 10.1038/s41467-024-48998-4 — doi.org/10.1038/s41467-024-48998-4
Zhang Z. et al. "A multimodal robotic platform for multi-element electrocatalyst discovery", Nature 647, 390–396 (2025), DOI 10.1038/s41586-025-09640-5 (abstract and Extended Data legends checked; the main text is behind a subscription) — doi.org/10.1038/s41586-025-09640-5
Ghafarollahi A., Buehler M. J. "SciAgents: Automating Scientific Discovery Through Bioinspired Multi-Agent Intelligent Graph Reasoning", Adv. Mater. 37, 2413523 (2025), DOI 10.1002/adma.202413523 (abstract checked in arXiv:2409.05556) — arxiv.org/abs/2409.05556
Ghafarollahi A., Buehler M. J. "AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence", arXiv:2407.10022 (2024) (preprint; a journal version was not found) — arxiv.org/abs/2407.10022
Ni Z. et al. "MatPilot: an LLM-enabled AI Materials Scientist under the Framework of Human-Machine Collaboration", arXiv:2411.08063 (2024) (preprint; a journal version was not found) — arxiv.org/abs/2411.08063
Mandal I. et al. "Evaluating large language model agents for automation of atomic force microscopy", Nat. Commun. 16, 9104 (2025), DOI 10.1038/s41467-025-64105-7 (abstract checked in arXiv:2501.10385) — arxiv.org/abs/2501.10385
Liu S. et al. "MatTools: Benchmarking Large Language Models for Materials Science Tools", arXiv:2505.10852 (2025) — arxiv.org/abs/2505.10852
Model Context Protocol "Specification" (revision checked: 2026-07-28) — modelcontextprotocol.io/specification/latest
Materials Project "Materials Project documentation" (describes it as a DOE effort to pre-compute properties of materials and make the data public) — docs.materialsproject.org/
Jain A. et al. "Commentary: The Materials Project: A materials genome approach to accelerating materials innovation", APL Mater. 1, 011002 (2013), DOI 10.1063/1.4812323 — doi.org/10.1063/1.4812323
NIST "CODATA Value: elementary charge" (2022 CODATA recommended values) — physics.nist.gov/cgi-bin/cuu/Value?e
NIST "CODATA Value: Avogadro constant" (2022 CODATA recommended values) — physics.nist.gov/cgi-bin/cuu/Value?na
ISO/IEC "ISO/IEC 22989:2022 Information technology — Artificial intelligence — Artificial intelligence concepts and terminology" (ISO page) — iso.org/standard/74296.html
ISO/IEC "ISO/IEC 23894:2023 Information technology — Artificial intelligence — Guidance on risk management" (ISO page) — iso.org/standard/77304.html
ISO/IEC "ISO/IEC 42001:2023 Information technology — Artificial intelligence — Management system" (ISO page) — iso.org/standard/42001
NIST "Artificial Intelligence Risk Management Framework (AI RMF 1.0)", NIST AI 100-1 (January 2023), DOI 10.6028/NIST.AI.100-1 — doi.org/10.6028/NIST.AI.100-1
NIST "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile", NIST AI 600-1 (26 July 2024), DOI 10.6028/NIST.AI.600-1 — doi.org/10.6028/NIST.AI.600-1

14. Claim-to-source audit

Claim in the text	Basis	Class
ChemCrow: 18 tools, GPT-4, autonomous synthesis of DEET and three known catalysts on RoboRXN, automatic revision of procedures, the statement that the tools could be used individually with likely the same result, the chromophore (target 369 nm, measured 336 nm), the disagreement between four experts and the LLM grader, the statement that a standardized evaluation technique is not yet established, the tool categories	Reference 1 https://doi.org/10.1038/s42256-024-00832-8	Sourced
Coscientist: the GPT-4 Planner and four commands, execution in an isolated container, OT-2 and Emerald Cloud Lab, Suzuki and Sonogashira reactions, GC-MS confirmation, plates moved by hand with no human decision-making, self-correction of a method name, non-browsing models getting ibuprofen wrong, the caveat on the comparison with Bayesian optimization, the dual-use study	Reference 2 https://doi.org/10.1038/s41586-023-06792-0	Sourced
LLaMP: hierarchical RAG-type ReAct agents, the Materials Project, no fine-tuning, reduced errors in bulk modulus, band gap and formation energy, errors seeming to derive from mixed data sources, the self-consistency metric	Reference 3 https://arxiv.org/abs/2401.17244	Sourced
MDCrow: 40 tools, 25 tasks, gpt-4o completing with low variance followed by llama3-405b, the effect of prompt style	Reference 4 https://arxiv.org/abs/2502.09565	Sourced
ChatMOF: three components; 96.9% for searching, 95.7% for predicting and 87.5% for generating (GPT-4)	Reference 5 https://doi.org/10.1038/s41467-024-48998-4	Sourced
CRESt: large multimodal models, knowledge-assisted Bayesian optimization, robots, cameras and a vision-language model, over 900 catalyst chemistries, 3,500 electrochemical tests, 3 months, an eight-element space, a 9.3-fold improvement in cost-specific performance (the authors' claim)	Reference 6 https://doi.org/10.1038/s41586-025-09640-5	Not yet confirmed
SciAgents (knowledge graphs and several agents; the authors' claim), AtomAgents (alloy design), MatPilot (human-AI collaboration, automated experiments); the last two are preprints with no journal version found	Reference 7 https://arxiv.org/abs/2409.05556 Reference 8 https://arxiv.org/abs/2407.10022 Reference 9 https://arxiv.org/abs/2411.08063	Not yet confirmed
AFMBench and AILA: even state-of-the-art models struggle with basic tasks, Claude 3.5 Sonnet strong at materials question answering yet poor as an agent, deviation from instructions, prompt fragility, multi-agent better than single-agent. MatTools: 69,225 questions, 49 tasks and 138 subtasks, three findings	Reference 10 https://arxiv.org/abs/2501.10385 Reference 11 https://arxiv.org/abs/2505.10852	Sourced
MCP: an open protocol connecting LLM applications with external data and tools, JSON-RPC 2.0, resources, prompts and tools, tools representing arbitrary code execution, explicit consent before invocation, the protocol itself not enforcing the principles, revision checked 2026-07-28	Reference 12 https://modelcontextprotocol.io/specification/latest	Sourced
The Materials Project as a DOE effort that pre-computes material properties and publishes them	Reference 13 https://docs.materialsproject.org/ Reference 14 https://doi.org/10.1063/1.4812323	Sourced
Elementary charge 1.602176634×10⁻¹⁹ C and Avogadro constant 6.02214076×10²³ /mol (CODATA 2022, exact)	Reference 15 https://physics.nist.gov/cgi-bin/cuu/Value?e Reference 16 https://physics.nist.gov/cgi-bin/cuu/Value?na	Sourced
Numbers, titles, status (published) and issue dates of ISO/IEC 22989:2022, 23894:2023 and 42001:2023 (July 2022, February 2023, December 2023); NIST AI 100-1 (January 2023) and NIST AI 600-1 (26 July 2024)	Reference 17 https://www.iso.org/standard/74296.html Reference 18 https://www.iso.org/standard/77304.html Reference 19 https://www.iso.org/standard/42001 Reference 20 https://doi.org/10.6028/NIST.AI.100-1 Reference 21 https://doi.org/10.6028/NIST.AI.600-1	Sourced
−0.52 eV/atom × 96.485 ≒ −50.2 kJ/mol and a factor of about 96; the chromophore gap 369 − 336 = 33 nm (about 9%)	Our calculation (Python)	Our calculation
The main text of CRESt; the number of experiments, detailed definitions of success and independent verification for Coscientist and ChemCrow; figures in the published versions of LLaMP, MDCrow, MatTools and AFMBench; the content of studies controlling materials equipment through MCP; model licences and repeat tests with the latest versions; an international standard specific to agents in materials research	Not confirmed in the primary sources reachable in this survey, so not stated	Commentary
The readings "the LLM only calls tools and people check validity", "gates, approval and equipment-side limits", "Levels 0-4", "what is easy to delegate and what needs expert judgment", "measure on your own tasks before adoption" and "make your own equipment into tools"	Our own arrangement and commentary based on published material; not a view stated by the researchers or institutions	Commentary
Figs. 2 to 8 are conceptual diagrams (Fig. 3 includes a calculation); Figs. 1 and 9 are slots for AI-generated images; the quadrant positions in Fig. 5 do not express merit	Note by this article	Commentary

Last updated: 22 October 2026. Sources are limited to primary material (peer-reviewed papers, arXiv papers, the official MCP specification, and ISO and NIST pages). Worked numbers are our own calculations, not results of an actual agent. The main text of CRESt, details of the number of experiments and definitions of success, independent verification, model licences and repeat tests with the latest versions, and an international standard specific to agents in materials research could not be confirmed and are not stated. Figs. 1 and 9 are slots for AI-generated images; the other figures are explanatory conceptual diagrams.