1. What a large language model is (the short version)

• It predicts the next token. An LLM takes a sequence of tokens, outputs a probability for the next token, picks one, appends it, and repeats to produce text (autoregression). In training, its parameters are adjusted so that it predicts the "next token" of a huge amount of text.
• The core is the Transformer. Attention (which tokens to look at) and a feed-forward network (FFN) are wrapped in residual connections and normalisation, and L such layers are stacked. The parameter count is set, roughly, by the number of layers and the square of the width (dimension) Sourced [Ref. 1, 11].
• "Size" is not one number. Total parameters, active parameters per token, context length (a declared value) and KV-cache size are separate quantities. For some models the scale has not been published Sourced [Ref. 17, 18, 21].

2. Language-model basics — next-token probability and autoregression

Treat text as a sequence of tokens x₁, x₂, …, x_T. A language model splits the probability of the whole sequence into a product of "the probability of each token given the tokens before it": p(x₁…x_T) = ∏ p(x_t | x₁…x_t−1). This is autoregression (this article's commentary). In the Transformer of Vaswani et al., the decoder masks attention so that a position cannot attend to later positions and offsets the output by one position, so the prediction for position i can depend only on the known tokens before i Sourced [Ref. 1].

View original SVG, values and source annotations

Besides the encoder-decoder form of the original paper (aimed at translation), Transformers come in encoder-only forms (such as BERT) and decoder-only forms (such as the GPT line). BERT was reported as a way of pre-training a bidirectional Transformer Sourced [Ref. 8]; GPT (2018) was a 12-layer decoder-only Transformer (768 dimensions, 12 heads, an FFN of 3,072 dimensions) Sourced [Ref. 6]. T5 was an encoder-decoder model that casts every task as "text to text" and was tried at up to 11 billion parameters Sourced [Ref. 9]. Most of today's conversational LLMs are decoder-only (this article's summary; check each model's report for its actual structure).

3. Tokenisation — what the model reads is not "characters"

What the model processes is not characters or words but the numbers of tokens, items in a vocabulary. Sennrich et al. proposed that encoding rare and unknown words as sequences of subword units lets a fixed vocabulary cover every word (open-vocabulary), and applied byte pair encoding (BPE) to this purpose Sourced [Ref. 3]. SentencePiece is a language-independent tokenizer that learns a subword vocabulary directly from raw text and handles both encoding and decoding Sourced [Ref. 4].

Vocabulary size is a design choice. Llama 3 uses a 128K vocabulary, made of 100K tokens from tiktoken plus 28K added to better support languages other than English, and reports that compression on a sample of English rose from 3.17 to 3.94 characters per token compared with the Llama 2 tokenizer Sourced [Ref. 14]. On the other hand, Petrov et al. report that the same text translated into different languages can differ greatly in token count, by up to 15 times in some cases, and that the gap persists even in tokenizers built with multilingual support in mind Sourced [Ref. 5].

4. Attention — following a small numerical example

Attention is the mechanism by which each token decides which other tokens to consult, and by how much, and mixes in their information. The original paper gives Attention(Q, K, V) = softmax(QKᵀ / √d_k) V: the dot products of queries Q with keys K are divided by √d_k, turned into weights by softmax, and used to average the values V Sourced [Ref. 1]. The division by √d_k is explained as a guard against large dot products pushing the softmax into a saturated region when the dimension is large [Ref. 1]. In the base model of the original paper, the model dimension is 512 and there are 8 heads, each computing attention through its own projections, with the results concatenated [Ref. 1].

• This is a minimal example with three tokens and Q = K. Applying the causal mask to the scores QKᵀ/√2 and taking the softmax gives the weights in the figure above.
• Token 1 can see only itself, so its weight is 1.00; token 3 puts 0.50 on itself and 0.25 on each of tokens 1 and 2. Each output is the weighted average of the V vectors in view (for token 3, (0.99, 2.51)).
• The weights shown are rounded to two decimal places. The outputs are computed from the unrounded values.

Our calculation. Python (NumPy), double precision. The vectors are a minimal illustration, not values from any real model.

(1) Position information

Attention by itself does not distinguish the order of tokens, so position information has to be added. The original paper uses sinusoidal positional encoding and says learned positional embeddings gave nearly identical results Sourced [Ref. 1]. Su et al. later proposed RoPE (rotary position embedding), which encodes absolute position with a rotation matrix while building a dependence on relative position into the self-attention formula Sourced [Ref. 27]. Llama 3 uses RoPE (θ = 500,000) [Ref. 14].

(2) Compute and memory: the square of the context length

Attention handles scores for every pair of tokens in the context (n × n), so its cost grows with the square of the context length n. Kaplan et al. estimate the compute of one forward pass as C_forward ≈ 2N + 2·n_layer·n_ctx·d_attn (N is the number of non-embedding parameters) and say the context-dependent term is small when d_model is well above n_ctx/12 Sourced [Ref. 11]. Putting Llama 3 405B's values into this formula (126 layers, d = 16,384, N ≈ 4.05×10¹¹), the context-dependent term is about 4% of 2N at n = 8,192 and about 67% at n = 131,072 (Our calculation: Kaplan's formula applied with the full n_ctx, ignoring savings from GQA and masking and any implementation details). At long contexts, attention can no longer be ignored.

Moreover, writing the n × n score matrix out to memory in the naive way gives 1.7×10¹⁰ elements at n = 131,072, which at half precision (2 bytes) is about 34 GB per head per layer (Our calculation). FlashAttention, by Dao et al., is an exact attention algorithm that avoids writing out this matrix: it works on tiles and reduces reads and writes between the GPU's high bandwidth memory (HBM) and on-chip SRAM Sourced [Ref. 30]. In short, the cost of attention is set not only by arithmetic but by data movement between memories.

5. The Transformer block, recent variants, and a parameter-count check

One block of a decoder-only Transformer consists of "normalise → attention → add the residual" and "normalise → FFN → add the residual". L such blocks are stacked, and a final stage outputs scores over the whole vocabulary.

Since the original paper, several components have been replaced. The table below sets out the differences from the original paper, including components the LLaMA report lists as improvements taken from earlier models Sourced [Ref. 12].

All Sourced (the original papers and technical reports). We do not judge which combination of components is best.

Checking the parameter count

Kaplan et al. estimate the number of non-embedding parameters in the standard setting (d_attn = d_ff/4 = d_model) as N ≈ 12·n_layer·d_model² Sourced [Ref. 11]. We checked this formula, and a component-by-component count, against published configurations.

Our calculation. "Counted by component" uses Llama 3's published values (layers, dimension, FFN dimension 14,336 / 28,672 / 53,248, attention heads 32 / 64 / 128, 8 KV heads, vocabulary 128,000, SwiGLU) [Ref. 14] and counts attention (the Q, K, V and output projections) and the FFN (three matrices). It assumes separate input and output embeddings, so vocabulary × dimension is added twice. Small terms such as layer normalisation are left out. The stated 8B / 70B / 405B are rounded names. GPT-3 uses the published values (96 layers, d = 12,288) [Ref. 10].

The check shows the following. For GPT-3, with its standard configuration, Kaplan's shortcut formula comes close (173.9 B against a stated 175.0 B). For Llama 3, which uses SwiGLU and GQA, the shortcut is off by a few per cent, and the component-by-component count is closer to the stated size (this article's commentary). Most of the parameters sit in the attention projection matrices and the FFN weight matrices, both large matrices that scale with the square of the dimension. The deeper and wider the model, the faster the amount of weights grows.

6. Pre-training — learning to predict the next token

In pre-training, the parameters are updated in the direction that minimises the cross-entropy loss (a loss based on the logarithm of the predicted probability) of the "next token" at every position in a large body of text (this article's commentary; Kaplan et al. also measure language-model performance by cross-entropy loss [Ref. 11]). The causal mask lets the predictions at all positions of one passage be computed in parallel (Section 2). Among published figures, the scale of training is as follows.

• GPT-3 (175B): all models were trained on a total of 300 billion tokens; the 175B model used a batch of 3.2 million tokens and a context length of 2,048 Sourced [Ref. 10].
• Llama 3: pre-trained on a corpus of about 15T multilingual tokens (15.6T for the flagship model), against 1.8T tokens for Llama 2. Pre-training compute for the flagship was 3.8×10²⁵ FLOPs, almost 50 times that of the largest Llama 2. It was first trained with a context of 8K tokens and then extended to 128K tokens Sourced [Ref. 14].
• DeepSeek-V3: the developer reports that full training took 2.788M H800 GPU hours Sourced (developer's stated value) [Ref. 18].

Training compute can be estimated roughly as "6 × parameters × training tokens". Kaplan et al. take the forward pass at about 2N per token and the backward pass at about twice that, so training costs about 6N FLOPs per token, and write total compute as C ≈ 6N·B·S (B the batch, S the number of updates) Sourced [Ref. 11]. We check this against published values.

Our calculation. 6ND is an approximation and excludes the context-length-dependent term of attention and any extra computation from the implementation. The report values are those stated in each paper. The relation between training data and compute is treated at length in our explainer on training data and scaling laws.

A pre-trained model writes continuations of text, but it is not yet well adjusted to follow instructions or behave safely. That adjustment (post-training) is covered in our explainer on post-training and fine-tuning. The Llama 3 report likewise describes several rounds of alignment with human feedback after pre-training [Ref. 14].

7. Autoregressive generation, sampling and the KV cache

(1) Picking one from the probabilities: sampling

In generation, the next token is chosen from the model's probabilities over the whole vocabulary. Decoding methods that maximise probability, such as picking the single most probable token (greedy) or searching over several candidates (beam search), can cause "degeneration": bland, incoherent output, or loops of repeated text. Holtzman et al. proposed nucleus sampling (top-p), which cuts off the unreliable tail of the distribution and samples from the set of tokens (the nucleus) that holds most of the probability Sourced [Ref. 32]. With settings such as temperature (a value that sharpens or flattens the distribution) and top-p, the same input can give different outputs (this article's commentary).

(2) The KV cache: remembering past K and V

When generating one token at a time, the K and V of past tokens do not change, so they are stored and reused instead of being recomputed each time. This is the KV cache. Pope et al. call the attention key and value tensors of each layer the KV cache and explain that it must be kept in memory for the duration of decoding Sourced [Ref. 31]. Shazeer observed that incremental inference is slow because of the memory-bandwidth cost of repeatedly loading the large key and value tensors, and proposed multi-query attention, in which keys and values are shared across heads Sourced [Ref. 28].

The size of the KV cache (per sequence) is 2 (K and V) × layers × KV heads × head dimension × bytes per element × number of tokens. Running Llama 3 405B (126 layers, 8 KV heads, head dimension 16,384 / 128 = 128) in bf16 (2 bytes) gives 516,096 bytes per token (about 0.52 MB), and about 67.6 GB (63 GiB) for one sequence of 128K (131,072) tokens. If the number of KV heads were the same as the number of attention heads, 128 (multi-head attention), the same length would need about 1.08 TB (Our calculation).

The weights are also large: 405B in bf16 is about 810 GB, 70B about 140 GB and 8B about 16 GB (Our calculation: parameters × 2 bytes, ignoring quantisation and compression). Pope et al. cite the fact that model parameters generally do not fit in the memory of a single accelerator chip as one reason inference is hard [Ref. 31].

8. Reading scale — dense models, mixture of experts, and undisclosed sizes

The table below sets out, as stated, the scale recorded in official papers and technical reports. The table does not let you read off which model is better, because the training data, the amount of training, the post-training and the evaluation settings all differ.

All Sourced (developers' stated values). B = billion. Context lengths as written in each source. Checked 15 October 2026. The table does not cover other models released since 2025, or the new generations of 2026.

Cautions in reading

• Total versus active: an MoE model must hold weights for all of its total parameters (memory), while the compute for one token scales with the active parameters (compute). Do not describe scale by one of them alone (this article's commentary).
• Context length is a declared value: the context lengths in the table declare what the design and training support. How well information can be used at long context needs separate evaluation, and there are reports of lower performance for information in the middle [Ref. 33].
• Treat "undisclosed" as undisclosed: the GPT-4 technical report does not state its scale [Ref. 21]. We do not estimate it here.

9. A lineage of major models (within official sources)

Arranged in broad lines, the flow around the Transformer looks like the figure below. Years are those of the paper or technical report. The arrows within a line show the order in time; they do not mean that one model improved on another or is better.

• The introduction of attention (2014): Bahdanau et al. proposed, for machine translation, a mechanism that (softly) searches for the relevant parts of the input at each decoding step Sourced [Ref. 2]. In 2017 the Transformer showed a structure built on attention alone, without recurrence or convolution [Ref. 1].
• Bigger pre-trained models (2018–2020): GPT (2018) [6], BERT (2018) [8], GPT-2 (2019, up to 1.5B) [7], T5 (2019) [9] and GPT-3 (2020, 175B) [10] widened scale and range of uses. The GPT-3 report examined performance in the few-shot setting, with no task-specific fine-tuning [10].
• The line with released weights (2023 onward): LLaMA is reported at 7B to 65B, Llama 2 at 7B to 70B and Llama 3 at 8B to 405B [12, 13, 14]. The LLaMA report stresses that it showed models trained only on publicly available data [12].
• The mixture-of-experts (MoE) line: a sparsely activated MoE layer (2017, up to 137 billion parameters) [15], the Switch Transformer (up to one trillion parameters) [16], then Mixtral [17], DeepSeek-V3 [18], Qwen3 [19] and gpt-oss [20].
• Models with undisclosed scale: the GPT-4 (2023) and Gemini 1.0 (2023) reports do not give, at least in part, the details of model structure or scale [21, 22].

This field changes generation quickly. As of the checking date of 15 October 2026, this article covers the principal official sources up to 2025. Models released in 2026 are not covered and have not been checked.

10. Limits, and what could not be confirmed / Summary

(1) Newer models

The table in this article covers the principal official sources up to 2025. We did not check models released in 2026 or the latest generations from each company. Model names, sizes and context lengths may change after the checking date (15 October 2026).

(2) Models whose scale is undisclosed

For GPT-4 and for Gemini 1.0 Ultra and Pro, among others, the scale is not written in official material. We do not use values from press reports or estimates.

(3) Benchmark figures

This article sets out the design and the way to read benchmarks, and gives no scores for individual models, so as not to rank figures obtained under different settings or by different evaluators.

(4) What is not covered

We do not cover the details of optimisation, initialisation, efficiency methods such as quantisation and distillation (our explainer on model efficiency), inference optimisation and serving (our explainer on inference optimisation and serving), or families other than the Transformer such as state-space models.

(5) Measuring tokenisers

We did not check primary sources for how many times more tokens a Japanese document needs than an English one, or for how chemical formulae and numbers are split.

(6) The texts of standards

The texts of ISO/IEC 22989, 23053, 42001 and 23894 are paid and we did not check them. We limit ourselves to confirming numbers, titles and publication dates.

11. How to read evaluations and standards

Evaluation of language models divides into the pre-training loss (perplexity), task benchmarks (knowledge, code, mathematics, long context and so on), comparison by human preference, and measurement of training and inference speed. In each case the number changes with the settings (number of shots, prompt, use of tools, sampling, who ran the evaluation), so read it together with the name, version and settings. Third-party leaderboards are used only as reference values, after checking their method and date. Safety and risk are not settled by numbers alone: standards and guidance (ISO/IEC, NIST) give frameworks for how organisations operate and evaluate. The table below sets out what we could confirm (checked 15 October 2026).

The arrangement of the table is Our calculation (organisation). ISO/IEC numbers and titles were checked on the official ISO pages, and NIST AI 600-1 in the NIST document (checked 15 October 2026). The texts of standards are paid, so we limit ourselves to titles and scope. The MLPerf versions are as shown on the pages on the checking date.

Cautions in reading (for materials engineers)

• Benchmark values change with settings and with who ran the evaluation: do not rank figures that differ in number of shots, prompt, use of tools, sampling, reasoning length or evaluator (the developer or a third party). A figure in a developer's technical report is the result under the settings that developer chose.
• Beware saturation and contamination: public benchmarks can saturate (differences at the top become hard to see) and can leak into training data (contamination) [Ref. 40, 42, 50]. Re-measuring on new problems, and measuring on your own tasks, is the basic remedy.
• Standards and guidance are mostly voluntary: ISO/IEC standards and NIST guidance generally gain binding force only when cited in law or contract, and NIST's Generative AI Profile is intended for voluntary use [Ref. 49] (the general position of ISO/IEC standards is this article's commentary).
• Declared values and effective values are different: context length, parameter count and speed figures are developers' stated values or measurements on a particular configuration. It is worth checking them with your own data and hardware.

12. Glossary

13. References

1. Vaswani A. et al. "Attention Is All You Need", arXiv:1706.03762 (NeurIPS 2017) — arxiv.org/abs/1706.03762
2. Bahdanau D., Cho K., Bengio Y. "Neural Machine Translation by Jointly Learning to Align and Translate", arXiv:1409.0473 — arxiv.org/abs/1409.0473
3. Sennrich R., Haddow B., Birch A. "Neural Machine Translation of Rare Words with Subword Units", arXiv:1508.07909 — arxiv.org/abs/1508.07909
4. Kudo T., Richardson J. "SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing", arXiv:1808.06226 — arxiv.org/abs/1808.06226
5. Petrov A. et al. "Language Model Tokenizers Introduce Unfairness Between Languages", arXiv:2305.15425 — arxiv.org/abs/2305.15425
6. Radford A. et al. (OpenAI) "Improving Language Understanding by Generative Pre-Training" (2018) — cdn.openai.com/.../language_understanding_paper.pdf
7. Radford A. et al. (OpenAI) "Language Models are Unsupervised Multitask Learners" (2019) — cdn.openai.com/.../language_models_are_unsupervised_multitask_learners.pdf
8. Devlin J. et al. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", arXiv:1810.04805 — arxiv.org/abs/1810.04805
9. Raffel C. et al. "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer", arXiv:1910.10683 — arxiv.org/abs/1910.10683
10. Brown T. B. et al. "Language Models are Few-Shot Learners", arXiv:2005.14165 — arxiv.org/abs/2005.14165
11. Kaplan J. et al. "Scaling Laws for Neural Language Models", arXiv:2001.08361 — arxiv.org/abs/2001.08361
12. Touvron H. et al. "LLaMA: Open and Efficient Foundation Language Models", arXiv:2302.13971 — arxiv.org/abs/2302.13971
13. Touvron H. et al. "Llama 2: Open Foundation and Fine-Tuned Chat Models", arXiv:2307.09288 — arxiv.org/abs/2307.09288
14. Grattafiori A. et al. (Llama Team, Meta) "The Llama 3 Herd of Models", arXiv:2407.21783 — arxiv.org/abs/2407.21783
15. Shazeer N. et al. "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer", arXiv:1701.06538 — arxiv.org/abs/1701.06538
16. Fedus W., Zoph B., Shazeer N. "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity", arXiv:2101.03961 — arxiv.org/abs/2101.03961
17. Jiang A. Q. et al. "Mixtral of Experts", arXiv:2401.04088 — arxiv.org/abs/2401.04088
18. DeepSeek-AI "DeepSeek-V3 Technical Report", arXiv:2412.19437 — arxiv.org/abs/2412.19437
19. Yang A. et al. (Qwen Team) "Qwen3 Technical Report", arXiv:2505.09388 — arxiv.org/abs/2505.09388
20. OpenAI "gpt-oss-120b & gpt-oss-20b Model Card", arXiv:2508.10925 — arxiv.org/abs/2508.10925
21. OpenAI "GPT-4 Technical Report", arXiv:2303.08774 — arxiv.org/abs/2303.08774
22. Gemini Team (Google) "Gemini: A Family of Highly Capable Multimodal Models", arXiv:2312.11805 — arxiv.org/abs/2312.11805
23. Chowdhery A. et al. "PaLM: Scaling Language Modeling with Pathways", arXiv:2204.02311 — arxiv.org/abs/2204.02311
24. Xiong R. et al. "On Layer Normalization in the Transformer Architecture", arXiv:2002.04745 (ICML 2020) — arxiv.org/abs/2002.04745
25. Zhang B., Sennrich R. "Root Mean Square Layer Normalization", arXiv:1910.07467 — arxiv.org/abs/1910.07467
26. Shazeer N. "GLU Variants Improve Transformer", arXiv:2002.05202 — arxiv.org/abs/2002.05202
27. Su J. et al. "RoFormer: Enhanced Transformer with Rotary Position Embedding", arXiv:2104.09864 — arxiv.org/abs/2104.09864
28. Shazeer N. "Fast Transformer Decoding: One Write-Head is All You Need", arXiv:1911.02150 — arxiv.org/abs/1911.02150
29. Ainslie J. et al. "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints", arXiv:2305.13245 — arxiv.org/abs/2305.13245
30. Dao T. et al. "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness", arXiv:2205.14135 — arxiv.org/abs/2205.14135
31. Pope R. et al. "Efficiently Scaling Transformer Inference", arXiv:2211.05102 — arxiv.org/abs/2211.05102
32. Holtzman A. et al. "The Curious Case of Neural Text Degeneration", arXiv:1904.09751 — arxiv.org/abs/1904.09751
33. Liu N. F. et al. "Lost in the Middle: How Language Models Use Long Contexts", arXiv:2307.03172 — arxiv.org/abs/2307.03172
34. Ji Z. et al. "Survey of Hallucination in Natural Language Generation", ACM Comput. Surv. 55 (2023), DOI 10.1145/3571730 — arxiv.org/abs/2202.03629
35. Hendrycks D. et al. "Measuring Massive Multitask Language Understanding", arXiv:2009.03300 — arxiv.org/abs/2009.03300
36. Srivastava A. et al. "Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models", arXiv:2206.04615 (TMLR) — arxiv.org/abs/2206.04615
37. Liang P. et al. "Holistic Evaluation of Language Models", arXiv:2211.09110 (TMLR 2023) — arxiv.org/abs/2211.09110
38. Chen M. et al. "Evaluating Large Language Models Trained on Code", arXiv:2107.03374 — arxiv.org/abs/2107.03374
39. Cobbe K. et al. "Training Verifiers to Solve Math Word Problems", arXiv:2110.14168 — arxiv.org/abs/2110.14168
40. Zhang H. et al. "A Careful Examination of Large Language Model Performance on Grade School Arithmetic", arXiv:2405.00332 — arxiv.org/abs/2405.00332
41. Chiang W.-L. et al. "Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference", arXiv:2403.04132 — arxiv.org/abs/2403.04132
42. Jacovi A. et al. "Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks", arXiv:2305.10160 — arxiv.org/abs/2305.10160
43. MLCommons "MLPerf Training" (benchmark list and rules summary; latest version shown on the checking date: v6.0) — mlcommons.org/benchmarks/training/
44. MLCommons "MLPerf Inference: Datacenter" (page shows v6.1 on the checking date) — mlcommons.org/benchmarks/inference-datacenter/
45. ISO/IEC "ISO/IEC 22989:2022 Information technology — Artificial intelligence — Artificial intelligence concepts and terminology" (ISO page) — iso.org/standard/74296.html
46. ISO/IEC "ISO/IEC 23053:2022 Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML)" (ISO page) — iso.org/standard/74438.html
47. ISO/IEC "ISO/IEC 42001:2023 AI management systems" (ISO page) — iso.org/standard/81230.html
48. ISO/IEC "ISO/IEC 23894:2023 AI — Guidance on risk management" (ISO page) — iso.org/standard/77304.html
49. NIST "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile", NIST AI 600-1, July 2024, DOI 10.6028/NIST.AI.600-1 — doi.org/10.6028/NIST.AI.600-1
50. Wang Y. et al. "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark", arXiv:2406.01574 — arxiv.org/abs/2406.01574

14. Claim-to-source audit

Last updated 15 October 2026. Sources are limited to primary material (the original arXiv papers, developers' technical reports and model cards, and the official pages of MLCommons, ISO and NIST). The texts of ISO/IEC standards are paid and could not be checked. Model names, sizes and context lengths are as stated in official material on the checking date; this field changes quickly, so please do not read old information as "current". Models released in 2026, the scale of models whose scale is undisclosed, per-model benchmark scores, measured token counts for Japanese text and chemical formulae, and the texts of ISO/IEC standards could not be confirmed and are not stated here. All figures are explanatory concept graphics or drawings that include our calculations. Figs. 1 and 6 are ImageGen-generated conceptual illustrations, and none of them shows an internal measurement of a real model or a physical product.