1. The short version / before you read / where this sits in the series

• AI drafts the "reading" of measured data. It is used to narrow down which crystals are in an XRD pattern (phase identification), to sort a micrograph by microstructure (classification), and to reduce noise Sourced [Ref. 1, 2, 11].
• Accuracy figures can only be read with their conditions. The same "about 90%" means different things depending on whether the data were calculated or measured, how many candidates there were, and whether the split was by sample. In the debate over an autonomous lab's analysis, a manual re-analysis later corrected the counts Sourced [Ref. 5].
• People and experiments do the final checking. AI is a tool that narrows candidates and gives a degree of confidence (a probability). The working assumption is that a person looks again, and adds measurements, when the probability is low or when the phase may not be among the candidates.

2. What "reading measured data" means — waveforms, images and AI's three jobs

Start with an everyday scene. You measure a powder sample by XRD and get a waveform of angle against intensity. What do you do with it? Most likely you compare it in your head with reference data: "this peak position belongs to that phase", "this small bump might be an impurity". With a micrograph, too, you tell regions apart from experience: "this white needle-like part is probably cementite".

This "comparing and telling apart" is the work handed to AI. There are two kinds of data: waveforms (one-dimensional) and images (two-dimensional).

The table is this article's own classification, for orientation.

View original SVG, values and sources

Original SVG reference: Concept diagram (vector drawing). How AI reads measured data. AI handles step 3; measurement (step 1) and the final judgment (step 4) remain human and experimental work. The grouping is this article's own arrangement.

3. Phase identification from XRD — conventional matching versus AI (CNN)

(1) First, why an XRD waveform works like a "fingerprint"

When X-rays hit a crystal, the regular arrangement of atoms produces strong reflections only at particular angles. The angle depends on the spacing d between planes in the crystal and the X-ray wavelength λ (Bragg's law, nλ = 2d sinθ). Because plane spacings differ from substance to substance, the waveform of angle against intensity (the diffraction pattern) acts as the crystal's "fingerprint" (our explanation).

Check it with numbers (Our calculation). The lattice parameter of silicon is about 5.431 Å (the value certified for the NIST reference material is 0.5431109 nm [Ref. 21]). The spacing of the (111) planes is 5.431/√3 ≈ 3.136 Å. With Cu Kα radiation (wavelength 1.5406 Å), sinθ = 1.5406/(2×3.136) gives θ ≈ 14.22°, so the measured angle (2θ) is 28.44°.

A "fingerprint match" is a helpful analogy. A fingerprint helps identify a person. But the analogy has limits: in a mixture, several fingerprints overlap, and where peaks coincide it can become unclear which phase a peak belongs to.

(2) The conventional way: extract peaks and match them

Conventional "phase identification" extracts peak positions and intensities from the measured pattern and compares them with the reference patterns in a standard database (for example the freely available Crystallography Open Database [Ref. 25], or paid databases). Software lists candidates in order of similarity, and a person makes the final choice. Szymanski et al. also use this kind of profile matching as the representative conventional method in their comparison [Ref. 2].

(3) The AI (CNN) way: feed in the whole waveform and output probabilities

The AI approach takes the whole waveform (for example 4,501 intensity values for 2θ from 10° to 80° [Ref. 2]) as input and outputs "how likely each candidate phase is". The model used is a convolutional neural network (CNN).

In the method of Szymanski et al., the prediction is repeated 1,000 times for the same input (randomly dropping some of the network's connections each time), and the fraction of repeats in which a phase is chosen is treated as its "probability" [Ref. 2]. This probability can serve as a cue for deciding which results a person should review. In that paper, for a test on calculated single-phase patterns, keeping only predictions with a probability above 70% (84% of the whole test set) raised accuracy from 92% to 96% Sourced [Ref. 2]. We could not confirm from the paper whether this probability is calibrated, that is, whether predicted probabilities match how often the answers are actually right.

More detail: why the candidate phases must be fixed in advance

These methods assume that the candidate answers (reference phases) are fixed beforehand. In the evaluation by Szymanski et al., the answer was chosen from 255 reference phases in the Li–Mn–Ti–O–F system (140 stable phases reported in experiments plus 115 hypothetical solid solutions) Sourced [Ref. 2]. If the sample contains a phase that is not among the candidates, the AI answers with "the closest one among the candidates". This is how the general weakness of machine learning at extrapolation shows up in XRD (our explanation).

4. Where the training data come from — calculated patterns and how they differ from measurements

To train AI, you need a large number of pairs of a waveform and its answer (which phase it is). For new materials and thin films, however, labelled measured data hardly exist. So many studies use diffraction patterns calculated from known crystal structures as training data. Oviedo et al. used calculated patterns made from a crystal structure database (ICSD) [Ref. 1], and Szymanski et al. did the same [Ref. 2].

Here is the pitfall. A calculated pattern is for an "ideal crystal", and measurements differ from it in three ways.

• Position shift: strain in a thin film, solid solution (a shift in composition) or a sample sitting at the wrong height moves the peak angles.
• Width change: the smaller the crystal grains (crystallites), the broader the peaks.
• Intensity change: when crystals align in a particular direction (texture), the ratios of peak heights change. This happens especially often in thin films [Ref. 1].

Szymanski et al. added such mismatches to the calculated patterns in advance to enlarge the training set (data augmentation): up to ±4% strain on the lattice parameters, peak widths corresponding to crystallite sizes of 1–100 nm (Scherrer equation), and up to ±50% change in peak intensities. They made 150 patterns per reference phase, 38,250 calculated spectra for 255 phases Sourced [Ref. 2]. Oviedo et al. likewise proposed "physics-informed data augmentation" that accounts for the orientation and lattice strain typical of thin films Sourced [Ref. 1].

The numbers under Fig. 4 (our calculation) give a feel for the size of the mismatches. A 1% change in the lattice parameter alone moves a peak near 28° by about 0.3°. An augmentation of ±4% corresponds to a shift of roughly ±1.1°. Szymanski et al. also report that augmentation cannot cope with changes beyond its range, such as large composition shifts. In their evaluation before adding hypothetical solid solutions to the training, the structure was identified correctly for 11 of 20 calculated patterns (95% after adding them) Sourced [Ref. 2].

5. Reading accuracy figures together with their conditions

"90% accuracy" looks like an easy number. But unless you know what counted as correct, what it was compared against, and what data it was measured on, it has no fixed meaning. Here we read figures from actual papers together with their conditions.

Let us read the figures in Fig. 5 one by one.

All figures are as reported in each paper. The table sets side by side figures whose conditions differ; it does not decide which method is better.

Letting AI decide the next measurement

In a separate paper, Szymanski et al. connected a diffractometer to AI and reported "adaptive XRD", which analyzes results during the measurement and decides which angles to measure carefully next. They showed that small amounts of a material could be detected with short measurement times, and that short-lived intermediate phases during a solid-state reaction could be followed on a standard in-house diffractometer Sourced [Ref. 3]. Combining measurement with AI is covered in our explainers on closed-loop optimization and on autonomous laboratories.

6. A case study: the debate over XRD analysis in an autonomous lab

Claims of accuracy are sometimes revisited later. Here we only set out, neutrally, what has been published. This is not an assessment of any person or research group.

This table is an arrangement of published documents only. We do not judge the merits of the criticism or the correction.

7. Spectroscopy (Raman and X-ray absorption)

AI analysis of waveforms is not limited to XRD. Here are two examples, with their conditions.

• Raman spectroscopy (measuring molecular vibrations of a substance): Liu et al. evaluated, on RRUFF (a public database of minerals [Ref. 9]), a method in which a CNN identifies substances from raw Raman spectra without preprocessing such as baseline correction. They report better classification performance than other machine learning, including support vector machines Sourced [Ref. 8]. However, the evaluation is on a mineral database, and we did not confirm specific accuracy figures, or performance on materials other than minerals, beyond what the abstract gives.
• X-ray absorption spectroscopy (XAS, which looks at the local structure around an atom): Torrisi et al. showed that random forests (a method that combines many decision trees) can predict the coordination number around an atom (4-, 5- or 6-fold), the nearest-neighbor distance and the Bader charge (the amount of electron charge assigned to an atom) from several thousand spectra of transition-metal oxides, and examined which spectral regions matter Sourced [Ref. 7]. The spectra used there were not measured but theoretical spectra calculated with FEFF (for computed structures from Materials Project and OQMD) [Ref. 7]. Whether the approach carries over to measured XAS was not tested in that work.

8. Microscopy images — classifying and painting microstructures

(1) Classifying microstructures: the steel example of Azimi et al.

Judging a steel's microstructure is work in which an experienced person looks at a micrograph and tells martensite from bainite and so on. Human judgments can contain subjectivity and observer-to-observer differences. Azimi et al. proposed a method that, from SEM images of low-carbon steel, judges the type of each pixel and decides the name of each object by majority vote (Fig. 6). For four microstructure constituents they report an accuracy of 93.94%, above the 48.89% of an earlier method on the same data Sourced [Ref. 11].

What to notice in this figure is how the images were split. In the paper, 11 images were used for training and 10 for testing; in objects, that was 2,831 for training and 2,262 for testing. The split was by image (objects from the same image do not straddle training and test), which prevents the leakage discussed in Section 10. The authors themselves call it a "proof of concept" and leave finer bainite classification (upper, lower and so on) to a next stage Sourced [Ref. 11].

(2) Organizing microstructure images as a "manifold": the ultrahigh-carbon steel of DeCost et al.

DeCost et al. released a dataset of SEM images of ultrahigh-carbon steel microstructures made with various heat treatments. They compared image features (from a method called SIFT, and from a CNN) and tried classifying the main constituent and the annealing condition. They also showed that a dimensionality-reduction method, t-SNE, can visualize the relation between microstructure and processing Sourced [Ref. 12]. A related paper from the same group [Ref. 27] released micrographs of ultrahigh-carbon steel made under various heat treatments together with processing metadata and source code, and showed tools for retrieving similar microstructures from an image and for drawing maps of microstructure and processing conditions Sourced. It is an example of sharing microstructure images together with their conditions.

(3) Atomic-resolution images: counting defects and following motion

Ziatdinov et al. used deep learning to read atom positions, atom types and defect types from scanning transmission electron microscopy (STEM) images in which atoms can be seen. With a "weakly supervised" method that finds even defects absent from the training data from atomic coordinates, they tracked the switching of the coordination of silicon dopant atoms in graphene Sourced [Ref. 10].

(4) Models for painting (segmentation), and pretraining

A representative model for painting images pixel by pixel is U-Net. It was designed to learn from few annotated images by using data augmentation heavily, and it was reported to beat the best prior method in an ISBI challenge on segmenting neuronal structures in electron microscopy stacks Sourced [Ref. 14]. For materials micrographs, "MicroNet" has also been reported: pretrained on more than 100,000 annotated micrographs of 54 material classes. Compared with models pretrained on ImageNet, it generalized better to images taken under different imaging conditions and was more accurate with less training data; when trained on a single image of a nickel superalloy, the relative intersection-over-union (IoU) error fell by 72.2% Sourced [Ref. 13].

9. Denoising — without removing what matters

Measured data carry noise. Reducing noise makes small peaks easier to find and gives AI cleaner input. But telling noise from signal is not simple. Here we check what happens with the simplest method, the moving average, using a small calculation.

(1) Our calculation: stronger smoothing flattens peaks

We add noise to a synthetic waveform (two close peaks of height 100 and 80, and a small peak of height 4) and apply a moving average with different window sizes. Fig. 8 shows the results.

With a small window (5 points), noise falls by about half (1.51 → 0.72) and the peak shapes are largely preserved. When the window equals the peak width (0.30°, 15 points), the taller peak has already lost about 20% of its height (100 → 82). With 41 points (0.82°), the two close peaks nearly become one lump, the valley fills in and the small peak is buried. Our calculation

The method matters even for the same window size. With the method proposed by Savitzky and Golay (fitting a polynomial inside the window and smoothing with it [Ref. 18]), in our calculation a 15-point window with a cubic polynomial kept the shapes well: the two peaks at 99 and 80, the valley at 10.8 and the small peak at 4.1 Our calculation. So a design decision is needed: "if you want to keep peak shapes, choose a method that has a shape-preserving property rather than a plain average" (our commentary).

(2) AI denoising: what is it learning?

Deep-learning denoising also decides "what is signal" from training data. A method that learns from just two noisy observations, without clean answers (Noise2Noise), has been reported Sourced [Ref. 16]. On the other hand, a review of deep learning for image reconstruction names, among the challenges, how to obtain training data, whether unknown structures can be discovered, and the danger of inferring details that are not really there Sourced [Ref. 17]. That review is about fluorescence microscopy, but the thinking also applies to electron micrographs and spectra (our commentary). A review [Ref. 15] gives an overview of deep learning in electron microscopy.

10. How data are split, and leakage — our calculation

The biggest point in reading the figures of Section 5 is whether the data used for evaluation are really different from the data used for training. Consider this scene. You measure the same sample six times. The six patterns are very similar. If five go into training and one into evaluation, the AI only has to identify a "sample it has already seen" and scores higher than its real ability. This is data leakage. In machine-learning-based science in general, leakage has been found in 17 fields, collectively affecting 294 papers Sourced [Ref. 19].

We check with a calculation. Assume three similar crystal types and make ten samples of each. Each sample has its peak positions shifted by ±0.7° (standard deviation) and is measured six times (180 patterns in all). The classification method is a simple one that answers the type of the nearest training pattern (nearest-neighbor). We compare two ways of splitting (Fig. 9).

• A: split measurement by measurement at random (measurements of the same sample can land in both training and test)
• B: split by sample (measurements of one sample go only to training or only to testing)

The result: with A, accuracy was 1.00 in all 30 runs; with B, the mean was 0.75 (0.48 to 1.00). Chance is 0.33. Our calculation. The figure for A is correct only "because the sample was seen during training" and says nothing about the ability on a new sample. Evaluating on "new samples", as in B, is the split closer to real use (measuring unknown samples).

11. A materials engineer's view — preparing your own data, and where people must decide

What can be left to AI, and where expert judgment is needed

The table is this article's own arrangement (commentary).

12. Limits, what could not be confirmed, and summary

(1) Figures whose original text we could not check

The 54% of Vecsei et al. rests on the description in the paper by Szymanski et al.; we did not check the original paper's text. We also did not confirm the specific accuracy of Liu et al. (Raman) beyond the abstract. Ziatdinov et al., Torrisi et al. and DeCost et al. are cited from their titles and abstracts.

(2) Texts of standards

For ISO 16700, ISO 643 and ISO/IEC TS 4213 we checked the number, title, status and scope on the ISO pages. The standards' texts (test conditions, procedures, acceptance criteria) are paid, and we did not check them. For Japanese Industrial Standards on X-ray diffraction analysis (such as JIS K 0131), we could not confirm the number, title or current edition from an official source on the checking date, so they are not stated.

(3) Data and methods not covered

We do not cover neutron diffraction, total scattering (PDF analysis), electron diffraction, EBSD (electron backscatter diffraction), spectral analysis other than deep learning (principal component analysis, non-negative matrix factorization and so on), or spectrum and image analysis with foundation models (very large models).

(4) Companies' products and software

We do not cover the products of instrument makers or analysis-software companies, or claims about their accuracy (we did not check them as primary sources).

(5) Developments in Japan

We did not survey data infrastructure or spectral databases at research institutes in Japan.

13. How to read the yardsticks and standards

How good AI analysis of spectra and images is depends first on four things: the basis of the correct answer (who decided it, and by what), how the evaluation data were split (by sample or by lot), the metric (accuracy, missed minor phases, IoU and so on) and what it is compared against (the baseline). Numbers can be compared only when these are aligned. There are international standards for AI in general, but we could not find a standard that sets "the conditions under which AI may be used for XRD phase identification". Here is what we could confirm (checked 22 October 2026).

The arrangement of the table is Our calculation (arrangement). Numbers, titles, editions and status for ISO and NIST were checked on their official pages (22 October 2026). The texts of standards are paid, so only their scope is noted.

Cautions when reading (for materials engineers)

• A ranking in a paper is not the ability on your samples: a paper gives results for the chemical system, instrument and conditions its authors chose. Re-measure with your own reference samples and real samples, split by sample and by lot.
• A prediction (AI's reading) is not experimental confirmation: AI output is "candidates and their likelihoods", not proof that a new phase exists. Design the work so another measurement backs it up (Section 6).
• If AI is trained on calculated data, evaluate on measurements too: a figure evaluated on calculated data depends on how the calculated data were made. Discount a figure that has no evaluation on measurements by that much.
• How strong a principle or standard is: the FAIR principles are recommendations, and ISO/IEC standards are mostly voluntary; they bind only when cited in law or a contract (the latter is our commentary). Plans and standards under revision are treated as Not yet confirmed.

14. Glossary

15. References

1. Oviedo F. et al. "Fast and interpretable classification of small X-ray diffraction datasets using data augmentation and deep neural networks", npj Comput. Mater. 5, 60 (2019), DOI 10.1038/s41524-019-0196-x (open access) — nature.com/articles/s41524-019-0196-x
2. Szymanski N. J., Bartel C. J., Zeng Y., Tu Q., Ceder G. "Probabilistic Deep Learning Approach to Automate the Interpretation of Multi-phase Diffraction Spectra", Chem. Mater. 33, 4204–4215 (2021), DOI 10.1021/acs.chemmater.1c01071 (the preprint arXiv:2103.16664 was used to check the text) — arxiv.org/abs/2103.16664
3. Szymanski N. J. et al. "Adaptively driven X-ray diffraction guided by machine learning for autonomous phase identification", npj Comput. Mater. 9, 31 (2023), DOI 10.1038/s41524-023-00984-y — nature.com/articles/s41524-023-00984-y
4. Szymanski N. J. et al. "An autonomous laboratory for the accelerated synthesis of inorganic materials", Nature 624, 86–91 (2023), DOI 10.1038/s41586-023-06734-w — nature.com/articles/s41586-023-06734-w
5. Szymanski N. J. et al. "Author Correction: An autonomous laboratory for the accelerated synthesis of inorganic materials", Nature 650, E1 (19 January 2026), DOI 10.1038/s41586-025-09992-y — nature.com/articles/s41586-025-09992-y
6. Leeman J. et al. "Challenges in High-Throughput Inorganic Materials Prediction and Autonomous Synthesis", PRX Energy 3, 011002 (2024), DOI 10.1103/PRXEnergy.3.011002 — doi.org/10.1103/PRXEnergy.3.011002
7. Torrisi S. B. et al. "Random forest machine learning models for interpretable X-ray absorption near-edge structure spectrum-property relationships", npj Comput. Mater. 6, 109 (2020), DOI 10.1038/s41524-020-00376-6 (open access) — nature.com/articles/s41524-020-00376-6
8. Liu J. et al. "Deep convolutional neural networks for Raman spectrum recognition: a unified solution", Analyst 142, 4067–4074 (2017), DOI 10.1039/c7an01371j — doi.org/10.1039/c7an01371j
9. RRUFF project "RRUFF Database of Mineral Data" (official site) — rruff.info
10. Ziatdinov M. et al. "Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations", ACS Nano 11, 12742–12752 (2017), DOI 10.1021/acsnano.7b07504 — doi.org/10.1021/acsnano.7b07504
11. Azimi S. M. et al. "Advanced Steel Microstructural Classification by Deep Learning Methods", Sci. Rep. 8, 2128 (2018), DOI 10.1038/s41598-018-20037-5 (open access) — nature.com/articles/s41598-018-20037-5
12. DeCost B. L., Francis T., Holm E. A. "Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures", Acta Mater. 133, 30–40 (2017), DOI 10.1016/j.actamat.2017.05.014 (arXiv:1702.01117 was used to check the abstract) — arxiv.org/abs/1702.01117
13. Stuckner J. et al. "Microstructure segmentation with deep learning encoders pre-trained on a large microscopy dataset", npj Comput. Mater. 8, 200 (2022), DOI 10.1038/s41524-022-00878-5 (open access) — doi.org/10.1038/s41524-022-00878-5
14. Ronneberger O., Fischer P., Brox T. "U-Net: Convolutional Networks for Biomedical Image Segmentation", arXiv:1505.04597 (MICCAI 2015) — arxiv.org/abs/1505.04597
15. Ede J. M. "Deep learning in electron microscopy", Mach. Learn.: Sci. Technol. 2, 011004 (2021), DOI 10.1088/2632-2153/abd614 — doi.org/10.1088/2632-2153/abd614
16. Lehtinen J. et al. "Noise2Noise: Learning Image Restoration without Clean Data", arXiv:1803.04189 (ICML 2018) — arxiv.org/abs/1803.04189
17. Belthangady C., Royer L. A. "Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction", Nat. Methods 16, 1215–1225 (2019), DOI 10.1038/s41592-019-0458-z — doi.org/10.1038/s41592-019-0458-z
18. Savitzky A., Golay M. J. E. "Smoothing and Differentiation of Data by Simplified Least Squares Procedures", Anal. Chem. 36, 1627–1639 (1964), DOI 10.1021/ac60214a047 — doi.org/10.1021/ac60214a047
19. Kapoor S., Narayanan A. "Leakage and the reproducibility crisis in machine-learning-based science", Patterns 4, 100804 (2023), DOI 10.1016/j.patter.2023.100804 (open access) — doi.org/10.1016/j.patter.2023.100804
20. Wilkinson M. D. et al. "The FAIR Guiding Principles for scientific data management and stewardship", Sci. Data 3, 160018 (2016), DOI 10.1038/sdata.2016.18 — doi.org/10.1038/sdata.2016.18
21. Cline J. P. et al. "Certification of Standard Reference Material 640g: Line Position and Line Shape Standard for Powder Diffraction (Silicon Powder)", NIST Special Publication 260-245 (February 2024), DOI 10.6028/NIST.SP.260-245 — doi.org/10.6028/NIST.SP.260-245
22. ISO "ISO 16700:2016 Microbeam analysis — Scanning electron microscopy — Guidelines for calibrating image magnification" (ISO page) — iso.org/standard/65375.html
23. ISO "ISO 643:2024 Steels — Micrographic determination of the apparent grain size" (ISO page) — iso.org/standard/82221.html
24. ISO/IEC "ISO/IEC TS 4213:2022 Information technology — Artificial intelligence — Assessment of machine learning classification performance" (ISO page) — iso.org/standard/79799.html
25. Gražulis S. et al. "Crystallography Open Database (COD): an open-access collection of crystal structures and platform for world-wide collaboration", Nucleic Acids Res. 40, D420–D427 (2012), DOI 10.1093/nar/gkr900 — doi.org/10.1093/nar/gkr900
26. Ede J. M. "Warwick electron microscopy datasets", Mach. Learn.: Sci. Technol. 1, 045003 (2020), DOI 10.1088/2632-2153/ab9c3c — doi.org/10.1088/2632-2153/ab9c3c
27. DeCost B. L. et al. "UHCSDB: UltraHigh Carbon Steel Micrograph DataBase", Integr. Mater. Manuf. Innov. 6, 197–205 (2017), DOI 10.1007/s40192-017-0097-0 — link.springer.com/article/10.1007/s40192-017-0097-0

16. Claim-to-source audit

Last updated 22 October 2026. Sources are limited to primary material (peer-reviewed papers, arXiv papers, and NIST and ISO official pages). The texts of ISO standards are paid and could not be checked. The calculation examples are on synthetic data that we generated, not results on real samples. The current edition of Japanese Industrial Standards such as JIS K 0131, the text of the original paper by Vecsei et al., the specific accuracy of the Raman work, and the calibration of the probabilities could not be confirmed and are not stated here. All figures are explanatory concept graphics or drawings that include our calculations. Figs. 1 and 7 are ImageGen-generated conceptual illustrations, and none of them shows a real instrument or measurement result.