1. The short version, what to know first, and where this fits in the series

• Bayesian optimization uses the results so far to choose the next experiment (or the next few) when every experiment is expensive. It has two parts: a guess about the response (a Gaussian process) and a rule for choosing what comes next (an acquisition function). The problems it is meant for typically allow at most a few hundred evaluations and about 20 or fewer input dimensions Sourced[Ref. 1]
• Bayesian optimization looks for the maximum; active learning improves the prediction overall. Both choose "the next point to measure" by using uncertainty. In a simple one-dimensional example of our own (our calculation), the share of runs that reached 98% or more of the true maximum after 10 experiments was 98% for Bayesian optimization and 29% for random choice Our calculation. But this is a made-up example, not a measure of real performance.
• In studies with experimental data, the benefit depends on conditions. There are cases where research is sped up by up to about 20 times compared with random choice, and cases where a poor choice of method makes it slower than random Sourced[Ref. 2]. The approach does not compete with design of experiments (DoE); the two can be combined.

2. Finding the best conditions with few experiments: three approaches

Suppose you want to fix the temperature, the time and the raw-material ratio for firing a material. With 10 values for each, that is 1,000 combinations. If one experiment takes a day, it takes about three years. How can you do fewer experiments?

The arrangement of the table is Our calculation (organisation). Commentary based on the explanation of DOE in the NIST/SEMATECH e-Handbook [Ref. 3].

How fast does a grid grow as conditions are added? A full set of two levels (high and low) takes 2ᵏ runs for k factors, and a full set of three levels takes 3ᵏ runs. The NIST e-Handbook explains that a three-level full factorial design takes 9 runs for 2 factors and 27 runs for 3 factors Sourced[Ref. 3]. Extending the count gives the table below.

The numbers in the table are Our calculation (powers; each added factor doubles or triples the count).

3. The Gaussian process: a "best-guess line" and a "band of not knowing"

The first part of Bayesian optimization is the Gaussian process. The name sounds hard, but what it does is this: "from the results at the points measured, it predicts the result at conditions not yet measured, together with a margin".

1. The best-guess line (mean): it passes through the results measured and joins the points smoothly.
2. The band of not knowing (uncertainty): narrow where there is data, wide far from the data.
3. The assumption: "if conditions are close, the results should be similar". How close counts as similar is set by the kernel (its length scale).

In Frazier's tutorial, the Gaussian process starts from a "prior guess" about the function (a mean and a kernel), adds the data, and updates the guess. The prediction for an unmeasured condition is a normal distribution, and if measurements have no noise, the mean passes through the measured points and the uncertainty there is zero Sourced[Ref. 1]. The left panel of the figure shows this case. For a review of Bayesian optimization as a whole, see Shahriari et al. Sourced[Ref. 4].

Real measurements scatter (the same condition gives slightly different results). As in the right panel, if noise is taken into account, the band does not shrink to zero even at the measured points, because "this single measurement may be a little off the true value". Criteria such as EI may need adjustment when there is noise Sourced[Ref. 1].

In more detail: why you can think of it as a "weighted average that favours nearby points"

The mean of a Gaussian-process prediction can be written as a weighted sum of the measured results. The weights are set by the "closeness" (the kernel) between the condition to predict and each measured point, and closer points count more. The guess line is therefore a smooth line that leans heavily on nearby results. In our example the kernel's length scale was fixed at 0.1 (with conditions scaled to 0-1). In practice it can also be estimated from data (our setting).

4. Where to measure next: acquisition functions, exploring and exploiting

The second part is the acquisition function, a formula that gives each candidate condition a score. The condition with the highest score is measured next. The difficulty is that two urges pull against each other.

• Exploitation: measure where the current guess says the result will be good.
• Exploration: measure where we do not yet know. A better spot may be hiding there.

In fishing terms, dropping a line at your usual spot is exploitation, and trying a new spot is exploration. The acquisition function balances the two in a single number. Two common ones follow.

• Expected improvement (EI): the expected amount by which the next measurement would beat the best result so far. It was proposed by Močkus and became widely known through the EGO (efficient global optimization) algorithm of Jones et al. It scores highly the places where the guess is high and the uncertainty is also large Sourced[Refs. 5, 1].
• Upper confidence bound (UCB): "guess (mean) + κ × not knowing (standard deviation)". A larger κ leans toward exploration. Srinivas et al. gave this method a theoretical guarantee Sourced[Ref. 6].

Checking with a small numerical example

Suppose the best result so far is 0.90. There are three candidates, with the following guess (μ) and uncertainty (σ). Which should be measured?

The numbers in the table are Our calculation (a made-up example). EI = (μ − best) × Φ(z) + σ × φ(z), with z = (μ − best) / σ, where Φ is the cumulative distribution of the standard normal and φ its density. PI = Φ(z).

Choosing by probability of improvement (PI) alone picks A, because A has the highest chance of beating the best. But both EI and UCB pick B: although its mean is low, its uncertainty is large and it has a chance of a big win. A different acquisition function can mean a different choice. As the note beside the figure says, even with the same data, EI and UCB can peak at different places.

In more detail: what the EI formula means

The first term of EI, (μ − best) × Φ(z), stands for "exploitation" (large when the mean is well above the best). The second term, σ × φ(z), stands for "exploration" (large when the uncertainty σ is large). EI is zero at points already measured (when there is no noise) Sourced[Ref. 1].

5. How it proceeds, step by step

Bayesian optimization repeats: build the model, choose the next condition, run the experiment, add the data. With each experiment the band narrows and the next point changes.

In the example, after the third experiment there is little data and the band is wide, so we do not yet know much. After the fifth, experiments have gone near a high region, while regions not yet measured also remain candidates. After the eighth, experiments have been placed at both the first peak found and a different peak. Rather than staying with a local good spot, going to check another peak is the benefit of exploration.

6. How much does it help?

"How many fewer experiments will Bayesian optimization really need?" is what a materials engineer most wants to know. Let us look at it in two ways.

(1) A simple simulation of our own (one dimension, no noise)

• We used a made-up one-dimensional response with two peaks (maximum 0.90), made the first three experiments random, and chose the rest by EI. The numbers are averages over 200 runs Our calculation.
• After 10 experiments, the share of runs reaching 98% or more of the true maximum was 98% for Bayesian optimization and 29% for random choice. After 15 it was 100% and 40% Our calculation.
• The mean best value at 10 experiments was about 0.89 for Bayesian optimization and about 0.80 for random choice Our calculation.

(2) Studies using experimental data

• Rohr et al. (2020): on electrochemical catalyst data (four chemical spaces of 2,121 catalysts each, the overpotential for the oxygen evolution reaction) they compared sequential learning (choosing the next point with a model) with random choice. They report that research "can be accelerated by up to a factor of 20" in specific scenarios, while "certain choices of models are ill-suited" and cause substantial deceleration compared with random acquisition Sourced[Ref. 2].
• Liang et al. (2021): on five experimental materials systems they compared Gaussian processes, random forests and others. They report that a Gaussian process with anisotropic kernels and a random forest performed comparably and both outperformed a Gaussian process with isotropic kernels, with the anisotropic-kernel Gaussian process the most robust Sourced[Ref. 8].

Here "acceleration" means: given a data set in which every result has already been measured, let the method choose, and measure how many times faster than random choice it finds good points. In a real search the answers are unknown, and there may be good points outside the data set.

7. Active learning: improving the overall prediction, not finding the maximum

Bayesian optimization aimed at "finding the maximum". Sometimes the aim is different, for example "to grasp the whole relationship between composition and property with few experiments". Then you use active learning.

Settles' literature survey gives the core idea of active learning as: a learning algorithm can reach higher accuracy with fewer labelled training examples if it is allowed to choose the data it learns from. For materials, a "label" is a measured result. A typical way to choose is uncertainty sampling (measure the point where the prediction is least confident) Sourced[Ref. 9]. Lookman et al. describe active learning in materials science as a combination of "a surrogate model that provides uncertainties" and "a utility function that prioritises the next points" Sourced[Ref. 10].

• In our example, the same made-up response was predicted from 15 experiments. The overall prediction error (RMSE) was 0.024 when measuring where the model knows least and 0.113 for random choice (mean of 200 runs) Our calculation.
• Measuring where we know least spreads the conditions over the whole range. Random choice can bunch up (right-hand side of the figure).
• Bayesian optimization concentrates on "the top of the hill"; active learning spreads out over "the whole range". Different aims lead to different choices.

The arrangement of the table is Our calculation (organisation). Commentary based on Settles [Ref. 9], Lookman et al. [Ref. 10] and Frazier [Ref. 1].

8. The relationship with design of experiments (DoE)

Design of experiments is a method with a long statistical tradition, widely used in materials and process work. The NIST e-Handbook describes design of experiments (DOE) as an efficient statistical procedure for planning experiments Sourced[Ref. 3]. How does it differ from Bayesian optimization, and how can they be combined?

The arrangement of the table is Our calculation (organisation). Based on the NIST e-Handbook [Ref. 3] (it describes the central composite and Box-Behnken designs as classical designs for quadratic models introduced in the 1950s, and says quadratic models are almost always sufficient for industrial applications) and on ISO 3534-3:2013 [Ref. 11] (terms for designs in which the response is one-dimensional and continuous, its expectation is linear in the parameters, and the error is normal with constant variance).

9. Pitfalls in practice

Bayesian optimization is handy, but it does not "work automatically once you put it in". Here are the points a materials engineer should check first.

• Noise (reproducibility): if results at the same condition scatter, you cannot tell whether "good" was due to the condition or chance. Repeat measurements at the same condition to learn the scatter, and treat it as noise Sourced[Ref. 1]. Lot-to-lot and operator differences are part of the noise.
• Dimensions (number of conditions): Frazier says successful applications are typically of about 20 dimensions or fewer, and lists methods that work well in high dimensions as an open problem Sourced[Ref. 1]. With many conditions, it is realistic to narrow to the influential factors with DoE before using it (inference).
• Constraints: safety, cost and conditions you cannot synthesise (for example a dangerous temperature) must be excluded from the search. Extensions that handle constraints are known Sourced[Ref. 1].
• Parallel experiments: when you fire several samples at once in a furnace, you decide "the next few" rather than "the next one". Parallel EI and the Constant Liar heuristic are described Sourced[Ref. 1].
• Defining the goal: if cost or another property matters as well as yield, you must decide "what to maximise". Extensions to several properties are being studied Sourced[Ref. 10].
• Choice of model: Liang et al. showed that the choice of kernel for the Gaussian process changes the result Sourced[Ref. 8]. A default setting is not always the best.

10. Cases with experiments

Here are some studies that connected Bayesian optimization or similar methods to experiments. All are reports (claims) by the research groups, under different conditions.

Each number is a value under the conditions of its own paper. The arrangement of the table is Our calculation (organisation). Checked on 22 October 2026.

Japanese research groups have also adapted Bayesian optimization to materials science. For example, COMBO (Ueno et al., 2016) is a Bayesian optimization library for materials science Sourced[Ref. 14]. The review by Terayama et al. frames the search in chemistry and materials as "black-box optimization", and describes combining methods such as Bayesian optimization, reinforcement learning and active learning with simulations based on quantum mechanics or molecular mechanics, and with experiments Sourced[Ref. 15].

11. A materials engineer's view

12. Limits, what we could not confirm, and a summary

(1) What we could not confirm

• From the abstract we could not confirm whether the study by Xue et al. (2016) included a control experiment against random or conventional methods.
• We did not confirm the detailed conditions of the comparison between humans and Bayesian optimization in Shields et al. (2021), such as the number of participants; we stay within the abstract.
• For the "about 20 times" of Rohr et al. (2020) we stay within the abstract and do not give the specific setting that produced it.
• Our simulation is a made-up one-dimensional, noise-free case; we did not compute results for many dimensions or with noise.
• We could not confirm an international standard specific to Bayesian optimization or active learning.

13. How to read the yardsticks and standards

The first yardstick of Bayesian optimization and active learning is the best value found per number of experiments (optimization), or the prediction error per number of experiments (active learning). Next come what it was compared with (random, humans, other methods) and on what data it was tried (real experiments, or a data set already measured in full). Checked on 22 October 2026.

The arrangement of the table is Our calculation (organisation). Numbers, titles and status were checked on the ISO page (22 October 2026). The text of the standard is paid, so we limit ourselves to its title and scope and do not state requirements.

Reading notes (for materials engineers)

• An "acceleration" figure is a value on a data set whose answers are known: in a real search the answers are unknown. Use the numbers as a rough guide when choosing a method [Refs. 2, 8]
• A search for the maximum tends to pick values that are high by chance: confirm the best point found with repeat experiments (commentary)
• Standards are recommendations or vocabulary: ISO 3534-3 is a vocabulary standard and does not guarantee a search procedure. Standards generally gain binding force only when cited in laws or contracts (commentary) [Ref. 11]

14. Glossary

15. References

1. Frazier P. I. "A tutorial on Bayesian optimization", arXiv:1807.02811 (2018) — arxiv.org/abs/1807.02811
2. Rohr B. et al. "Benchmarking the acceleration of materials discovery by sequential learning", Chem. Sci. 11, 2696–2706 (2020), DOI 10.1039/C9SC05999G — doi.org/10.1039/C9SC05999G
3. NIST/SEMATECH "e-Handbook of Statistical Methods, Chapter 5 "Process Improvement" (design of experiments, response surface designs)", (NIST Information Technology Laboratory) — itl.nist.gov/div898/handbook/pri/pri.htm
4. Shahriari B. et al. "Taking the human out of the loop: a review of Bayesian optimization", Proc. IEEE 104, 148–175 (2016), DOI 10.1109/JPROC.2015.2494218 — doi.org/10.1109/JPROC.2015.2494218
5. Jones D. R., Schonlau M., Welch W. J. "Efficient global optimization of expensive black-box functions", J. Global Optim. 13, 455–492 (1998), DOI 10.1023/A:1008306431147 — doi.org/10.1023/A:1008306431147
6. Srinivas N. et al. "Gaussian process optimization in the bandit setting: no regret and experimental design", ICML 2010, arXiv:0912.3995 — arxiv.org/abs/0912.3995
7. Häse F. et al. "Olympus: a benchmarking framework for noisy optimization and experiment planning", Mach. Learn.: Sci. Technol. 2, 035021 (2021), DOI 10.1088/2632-2153/abedc8 — doi.org/10.1088/2632-2153/abedc8
8. Liang Q. et al. "Benchmarking the performance of Bayesian optimization across multiple experimental materials science domains", npj Comput. Mater. 7, 188 (2021), DOI 10.1038/s41524-021-00656-9 — doi.org/10.1038/s41524-021-00656-9
9. Settles B. "Active learning literature survey", Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009) — minds.wisconsin.edu/handle/1793/60660
10. Lookman T. et al. "Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design", npj Comput. Mater. 5, 21 (2019), DOI 10.1038/s41524-019-0153-8 — doi.org/10.1038/s41524-019-0153-8
11. ISO "ISO 3534-3:2013 Statistics — Vocabulary and symbols — Part 3: Design of experiments", (ISO page) — iso.org/standard/44245.html
12. Xue D. et al. "Accelerated search for materials with targeted properties by adaptive design", Nat. Commun. 7, 11241 (2016), DOI 10.1038/ncomms11241 — doi.org/10.1038/ncomms11241
13. Shields B. J. et al. "Bayesian reaction optimization as a tool for chemical synthesis", Nature 590, 89–96 (2021), DOI 10.1038/s41586-021-03213-y — doi.org/10.1038/s41586-021-03213-y
14. Ueno T. et al. "COMBO: an efficient Bayesian optimization library for materials science", Mater. Discov. 4, 18–21 (2016), DOI 10.1016/j.md.2016.04.001 — doi.org/10.1016/j.md.2016.04.001
15. Terayama K., Sumita M., Tamura R., Tsuda K. "Black-box optimization for automated discovery", Acc. Chem. Res. 54, 1334–1346 (2021), DOI 10.1021/acs.accounts.0c00713 — doi.org/10.1021/acs.accounts.0c00713
16. Wilkinson M. D. et al. "The FAIR guiding principles for scientific data management and stewardship", Sci. Data 3, 160018 (2016), DOI 10.1038/sdata.2016.18 — doi.org/10.1038/sdata.2016.18

16. Claim-to-source audit

Last updated: 22 October 2026. Sources are limited to primary material (peer-reviewed papers, arXiv papers, and official NIST and ISO pages). The simulations and numerical examples (the curves in the figures, the acquisition-function table, the run counts against the number of factors) are our own calculations, not results for real materials. We could not confirm whether Xue et al. had a control experiment, the detailed conditions of the Shields et al. comparison, the specific setting behind the "factor of 20" of Rohr et al., simulations in many dimensions with noise, or an international standard specific to Bayesian optimization or active learning, so we do not state them. All figures are concept diagrams or diagrams that include our own calculation; the ImageGen images are conceptual illustrations.