NLP Lineage

31 August 2015 · Language modelling

BPE tokenization

Neural Machine Translation of Rare Words with Subword Units

R. Sennrich, B. Haddow, A. Birch

Date
31 August 2015
What the date marks
arXiv submission
Kind
Technique
Organisation
University of Edinburgh
Access
MIT
Other dates
ACL 2016
Threads
In the guided tour
Chapter 2. Words become vectors
Learning phases
1. Transformer architecture

A milestone: 3 later milestones descend directly from it, and its descendants reach 10 of the 10 threads. How levels are decided

Read the primary source (arxiv.org)

Full summary in progress. Until then, this page shows the milestone's record from the research data.

What it changed

Before it

Models had a fixed word vocabulary and an unknown token for everything else.

The problem

Rare words, names, morphology and other languages fell outside the vocabulary entirely.

What it introduced

Byte-pair encoding: merge frequent character pairs to build a subword vocabulary that can spell anything.

Why it matters

This is why your tokenizer splits words into pieces, and why models count letters badly and handle numbers oddly.

What modern models inherit

Subword tokenization, still the input layer of essentially every LLM.

What came next

WordPiece in GNMT, SentencePiece, and the byte-level BPE in GPT-2 onwards.

Before and after

The milestones it descends from, and the milestones that descend from it. Work developed alongside it is not counted. Weak links are left out unless you include them.

Before: where it came from

No documented ancestors are recorded for BPE tokenization without weak links.

No documented ancestors are recorded for BPE tokenization without weak links.

No recorded ancestors are recorded for BPE tokenization.

No recorded ancestors are recorded for BPE tokenization.

After: what it led to

BPE tokenization has 124 documented descendants across 11 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing its direct descendants and the 13 turning points among them.

BPE tokenization has 124 documented descendants across 11 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing all of them.

BPE tokenization has 131 recorded descendants, including indirect ones, across 11 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing its direct descendants and the 13 turning points among them.

BPE tokenization has 131 recorded descendants, including indirect ones, across 11 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing all of them.

  1. 2015BPE tokenizationThis milestone.
  2. 2019GPT-2 Turning pointbuilt directly on BPE tokenizationChapter 4. Pretrain once, use everywhere
  3. 2019RoBERTabuilt directly on BPE tokenizationChapter 4. Pretrain once, use everywhere
  4. 2019Megatron-LMwas influenced by GPT-2Chapter 4. Pretrain once, use everywhere
  5. 2019RLHF for LMsbuilt directly on GPT-2Chapter 6. Teaching models what people want
  6. 2019T5 Turning pointwas influenced by GPT-2Chapter 4. Pretrain once, use everywhere
  7. 2020Scaling laws Turning pointwas influenced by GPT-2Chapter 5. Scale becomes a science
  8. 2020GPT-3 Turning pointbuilt directly on GPT-2Chapter 5. Scale becomes a science
  9. 2020RLHF for summarisation Turning pointbuilt directly on RLHF for LMsChapter 6. Teaching models what people want
  10. 2020The Pilewas influenced by GPT-3Chapter 5. Scale becomes a science
  11. 2021Switch Transformerbuilt directly on T5Chapter 5. Scale becomes a science
  12. 2021CLIPbuilt directly on GPT-2Chapter 11. Seeing and hearing
  13. 2021LoRAbuilt directly on GPT-3Chapter 9. Making it run anywhere
  14. 2021Copilotwas influenced by Codex / HumanEvalChapter 7. The assistant goes public
  15. 2021Codex / HumanEvalbuilt directly on GPT-3Chapter 13. Models that act
  16. 2021FLANwas influenced by GPT-3
    was influenced by T5    Chapter 6. Teaching models what people want
  17. 2021T0built directly on T5Chapter 6. Teaching models what people want
  18. 2021HHH assistantwas influenced by RLHF for summarisationChapter 6. Teaching models what people want
  19. 2021WebGPTbuilt directly on GPT-3Chapter 13. Models that act
  20. 2022MT-NLGwas influenced by Scaling laws
    built directly on Megatron-LM    Chapter 5. Scale becomes a science
  21. 2022Chain of thought Turning pointwas influenced by GPT-3Chapter 12. Thinking longer
  22. 2022InstructGPT Turning pointbuilt directly on GPT-3Chapter 6. Teaching models what people want
  23. 2022Self-consistencybuilt directly on Chain of thoughtChapter 12. Thinking longer
  24. 2022STaRwas influenced by Chain of thoughtChapter 12. Thinking longer
  25. 2022Chinchilla Turning pointbuilt directly on Scaling lawsChapter 5. Scale becomes a science
  26. 2022PaLMwas influenced by Chain of thoughtChapter 5. Scale becomes a science
  27. 2022HH-RLHFbuilt directly on RLHF for summarisationChapter 6. Teaching models what people want
  28. 2022Flamingowas influenced by GPT-3
    built directly on CLIP    Chapter 11. Seeing and hearing
  29. 2022OPT-175Bbuilt directly on GPT-3
    was influenced by Megatron-LM    Chapter 8. The weights get out
  30. 2022Zero-shot CoTwas influenced by GPT-3Chapter 12. Thinking longer
  31. 2022Emergent abilitiesbuilt directly on Scaling lawsChapter 5. Scale becomes a science
  32. 2022BLOOMwas influenced by The PileChapter 8. The weights get out
  33. 2022LLM.int8()was influenced by OPT-175BChapter 9. Making it run anywhere
  34. 2022ReActbuilt directly on Chain of thought
    was influenced by WebGPT    Chapter 13. Models that act
  35. 2022Flan-T5 / Flan-PaLMbuilt directly on T5Chapter 6. Teaching models what people want
  36. 2022GPTQwas influenced by LLM.int8()Chapter 9. Making it run anywhere
  37. 2022GPT-3.5 Turning pointbuilt directly on GPT-3Chapter 7. The assistant goes public
  38. 2022ChatGPTbuilt directly on InstructGPT
    built directly on GPT-3.5    Chapter 7. The assistant goes public
  39. 2022Constitutional AIwas influenced by Chain of thoughtChapter 6. Teaching models what people want
  40. 2022Self-Instructbuilt directly on GPT-3Chapter 6. Teaching models what people want
  41. 2023Toolformerbuilt directly on GPT-3Chapter 13. Models that act
  42. 2023LLaMA Turning pointbuilt directly on BPE tokenizationChapter 8. The weights get out
  43. 2023llama.cppwas influenced by LLaMAChapter 9. Making it run anywhere
  44. 2023Alpacabuilt directly on LLaMAChapter 8. The weights get out
  45. 2023Claudebuilt directly on Constitutional AI
    was influenced by HH-RLHF    Chapter 7. The assistant goes public
  46. 2023GPT-4 Turning pointbuilt directly on Scaling lawsChapter 7. The assistant goes public
  47. 2023Reflexionbuilt directly on ReActChapter 13. Models that act
  48. 2023AutoGPTwas influenced by GPT-4Chapter 13. Models that act
  49. 2023Pythiawas influenced by Scaling lawsChapter 8. The weights get out
  50. 2023LLaVAbuilt directly on LLaMAChapter 11. Seeing and hearing
  51. 2023Emergence miragesucceeded Emergent abilitiesChapter 5. Scale becomes a science
  52. 2023100K contextbuilt directly on ClaudeChapter 10. Reading more than you remember
  53. 2023Tree of thoughtsbuilt directly on Chain of thoughtChapter 12. Thinking longer
  54. 2023QLoRAbuilt directly on LLaMAChapter 9. Making it run anywhere
  55. 2023DPObuilt directly on RLHF for summarisation
    built directly on InstructGPT    Chapter 6. Teaching models what people want
  56. 2023Process supervisionbuilt directly on Chain of thoughtChapter 12. Thinking longer
  57. 2023phi-1was influenced by GPT-3.5Chapter 8. The weights get out
  58. 2023Position interpolationbuilt directly on LLaMAChapter 10. Reading more than you remember
  59. 2023Llama 2 Turning pointbuilt directly on LLaMAChapter 8. The weights get out
  60. 2023GPT-4Vbuilt directly on GPT-4Chapter 11. Seeing and hearing
  61. 2023SWE-benchwas influenced by GPT-4
    built directly on Codex / HumanEval    Chapter 13. Models that act
  62. 2023Gemini 1.0succeeded PaLMChapter 11. Seeing and hearing
  63. 2023Mixtral 8x7Bbuilt directly on Switch TransformerChapter 8. The weights get out
  64. 2024DeepSeek LLMbuilt directly on Scaling laws
    was influenced by Llama 2    Chapter 8. The weights get out
  65. 2024DeepSeekMoEwas influenced by Switch TransformerChapter 8. The weights get out
  66. 2024Dolmawas influenced by The PileChapter 8. The weights get out
  67. 2024OLMowas influenced by Llama 2Chapter 8. The weights get out
  68. 2024GRPOwas influenced by InstructGPT
    built directly on DeepSeek LLM    Chapter 12. Thinking longer
  69. 2024Gemini 1.5built directly on Gemini 1.0Chapter 10. Reading more than you remember
  70. 2024Gemmawas influenced by Llama 2Chapter 8. The weights get out
  71. 2024Claude 3was influenced by GPT-4Chapter 14. Where the threads meet now
  72. 2024Chatbot Arenawas influenced by ChatGPTChapter 14. Where the threads meet now
  73. 2024Devinwas influenced by GPT-4Chapter 13. Models that act
  74. 2024Llama 3built directly on Llama 2Chapter 8. The weights get out
  75. 2024Phi-3built directly on phi-1Chapter 8. The weights get out
  76. 2024DeepSeek-V2 / MLAbuilt directly on DeepSeekMoEChapter 9. Making it run anywhere
  77. 2024SWE-agentbuilt directly on SWE-bench
    was influenced by ReAct    Chapter 13. Models that act
  78. 2024GPT-4obuilt directly on GPT-4Chapter 11. Seeing and hearing
  79. 2024Claude 3.5 Sonnetbuilt directly on Claude 3Chapter 14. Where the threads meet now
  80. 2024FineWebwas influenced by The PileChapter 8. The weights get out
  81. 2024Llama 3.1 405Bwas influenced by Position interpolationChapter 8. The weights get out
  82. 2024Test-time computewas influenced by Process supervision
    built directly on Self-consistency    Chapter 12. Thinking longer
  83. 2024o1 Turning pointwas influenced by Chain of thoughtChapter 12. Thinking longer
  84. 2024Computer usewas influenced by GPT-4VChapter 13. Models that act
  85. 2024Gemini 2.0built directly on Gemini 1.5Chapter 14. Where the threads meet now
  86. 2024o3 announcedbuilt directly on o1Chapter 12. Thinking longer
  87. 2024DeepSeek-V3was influenced by Llama 3.1 405BChapter 8. The weights get out
  88. 2025DeepSeek-R1was influenced by Chain of thoughtChapter 12. Thinking longer
  89. 2025Deep researchwas influenced by o3 releasedChapter 13. Models that act
  90. 2025Claude 3.7was influenced by o1Chapter 12. Thinking longer
  91. 2025Claude Codewas influenced by Computer use
    was influenced by Claude 3.7    Chapter 13. Models that act
  92. 2025GPT-4.5was influenced by Scaling lawsChapter 14. Where the threads meet now
  93. 2025Gemini 2.5was influenced by o1Chapter 12. Thinking longer
  94. 2025Llama 4succeeded Llama 3.1 405B
    was influenced by Switch Transformer    Chapter 8. The weights get out
  95. 2025o3 releasedbuilt directly on o1Chapter 12. Thinking longer
  96. 2025Qwen3was influenced by DeepSeek-R1Chapter 8. The weights get out
  97. 2025Claude 4succeeded Claude 3
    was influenced by SWE-bench    Chapter 13. Models that act
  98. 2025Kimi K2was influenced by DeepSeekMoE
    was influenced by GRPO    Chapter 8. The weights get out
  99. 2025IMO goldbuilt directly on Gemini 2.5
    was influenced by Test-time compute    Chapter 12. Thinking longer
  100. 2025gpt-ossbuilt directly on o3 releasedChapter 8. The weights get out
  101. 2025GPT-5succeeded GPT-4oChapter 14. Where the threads meet now
  102. 2025DeepSeek-V3.2built directly on DeepSeek-V3Chapter 10. Reading more than you remember
  103. 2025Claude Sonnet 4.5built directly on Claude 4Chapter 13. Models that act
  104. 2025IMO-ProofBenchwas influenced by IMO goldChapter 12. Thinking longer
  105. 2025GPT-5.1built directly on GPT-5Chapter 14. Where the threads meet now
  106. 2025Gemini 3built directly on Gemini 2.5Chapter 14. Where the threads meet now
  107. 2025Claude Opus 4.5was influenced by SWE-benchChapter 13. Models that act
  108. 2025GPT-5.2built directly on GPT-5.1Chapter 14. Where the threads meet now
  109. 2026Claude Opus 4.6built directly on Claude Opus 4.5Chapter 14. Where the threads meet now
  110. 2026GPT-5.3-Codexbuilt directly on GPT-5.2Chapter 13. Models that act
  111. 2026Gemini 3 Deep Thinkbuilt directly on Gemini 3
    was influenced by IMO gold    Chapter 12. Thinking longer
  112. 2026Qwen3.5built directly on Qwen3Chapter 8. The weights get out
  113. 2026Gemini 3.1built directly on Gemini 3Chapter 14. Where the threads meet now
  114. 2026GPT-5.4built directly on GPT-5.3-CodexChapter 14. Where the threads meet now
  115. 2026Muse Sparksucceeded Llama 4Chapter 8. The weights get out
  116. 2026Claude Opus 4.7built directly on Claude Opus 4.6Chapter 14. Where the threads meet now
  117. 2026GPT-5.5built directly on GPT-5.4Chapter 14. Where the threads meet now
  118. 2026DeepSeek-V4was influenced by DeepSeek-V3Chapter 8. The weights get out
  119. 2026Claude Opus 4.8built directly on Claude Opus 4.7Chapter 14. Where the threads meet now
  120. 2026Claude Sonnet 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  121. 2026GPT-5.6 Solbuilt directly on GPT-5.5Chapter 14. Where the threads meet now
  122. 2026Kimi K3built directly on Kimi K2Chapter 8. The weights get out
  123. 2026Claude Opus 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  124. 2026Qwen3.8built directly on Qwen3.5Chapter 8. The weights get out
  125. 2026Claude Opus 5.5built directly on Claude Opus 5Chapter 14. Where the threads meet now
  1. 2015BPE tokenizationThis milestone.
  2. 2019GPT-2 Turning pointbuilt directly on BPE tokenizationChapter 4. Pretrain once, use everywhere
  3. 2019RoBERTabuilt directly on BPE tokenizationChapter 4. Pretrain once, use everywhere
  4. 2019Megatron-LMwas influenced by GPT-2Chapter 4. Pretrain once, use everywhere
  5. 2019RLHF for LMsbuilt directly on GPT-2Chapter 6. Teaching models what people want
  6. 2019T5 Turning pointwas influenced by GPT-2Chapter 4. Pretrain once, use everywhere
  7. 2020Scaling laws Turning pointwas influenced by GPT-2Chapter 5. Scale becomes a science
  8. 2020GPT-3 Turning pointbuilt directly on GPT-2Chapter 5. Scale becomes a science
  9. 2020RLHF for summarisation Turning pointbuilt directly on RLHF for LMsChapter 6. Teaching models what people want
  10. 2020The Pilewas influenced by GPT-3Chapter 5. Scale becomes a science
  11. 2021Switch Transformerbuilt directly on T5Chapter 5. Scale becomes a science
  12. 2021CLIPbuilt directly on GPT-2Chapter 11. Seeing and hearing
  13. 2021LoRAbuilt directly on GPT-3Chapter 9. Making it run anywhere
  14. 2021Copilotwas influenced by Codex / HumanEvalChapter 7. The assistant goes public
  15. 2021Codex / HumanEvalbuilt directly on GPT-3Chapter 13. Models that act
  16. 2021FLANwas influenced by GPT-3
    was influenced by T5    Chapter 6. Teaching models what people want
  17. 2021T0built directly on T5Chapter 6. Teaching models what people want
  18. 2021HHH assistantwas influenced by RLHF for summarisationChapter 6. Teaching models what people want
  19. 2021WebGPTbuilt directly on GPT-3Chapter 13. Models that act
  20. 2022MT-NLGwas influenced by Scaling laws
    built directly on Megatron-LM    Chapter 5. Scale becomes a science
  21. 2022Chain of thought Turning pointwas influenced by GPT-3Chapter 12. Thinking longer
  22. 2022InstructGPT Turning pointbuilt directly on GPT-3Chapter 6. Teaching models what people want
  23. 2022Self-consistencybuilt directly on Chain of thoughtChapter 12. Thinking longer
  24. 2022STaRwas influenced by Chain of thoughtChapter 12. Thinking longer
  25. 2022Chinchilla Turning pointbuilt directly on Scaling lawsChapter 5. Scale becomes a science
  26. 2022PaLMwas influenced by Chain of thoughtChapter 5. Scale becomes a science
  27. 2022HH-RLHFbuilt directly on RLHF for summarisationChapter 6. Teaching models what people want
  28. 2022Flamingowas influenced by GPT-3
    built directly on CLIP    Chapter 11. Seeing and hearing
  29. 2022OPT-175Bbuilt directly on GPT-3
    was influenced by Megatron-LM    Chapter 8. The weights get out
  30. 2022Zero-shot CoTwas influenced by GPT-3Chapter 12. Thinking longer
  31. 2022Emergent abilitiesbuilt directly on Scaling lawsChapter 5. Scale becomes a science
  32. 2022BLOOMwas influenced by The PileChapter 8. The weights get out
  33. 2022LLM.int8()was influenced by OPT-175BChapter 9. Making it run anywhere
  34. 2022ReActbuilt directly on Chain of thought
    was influenced by WebGPT    Chapter 13. Models that act
  35. 2022Flan-T5 / Flan-PaLMbuilt directly on T5Chapter 6. Teaching models what people want
  36. 2022GPTQwas influenced by LLM.int8()Chapter 9. Making it run anywhere
  37. 2022GPT-3.5 Turning pointbuilt directly on GPT-3Chapter 7. The assistant goes public
  38. 2022ChatGPTbuilt directly on InstructGPT
    built directly on GPT-3.5    Chapter 7. The assistant goes public
  39. 2022Constitutional AIwas influenced by Chain of thoughtChapter 6. Teaching models what people want
  40. 2022Self-Instructbuilt directly on GPT-3Chapter 6. Teaching models what people want
  41. 2023Toolformerbuilt directly on GPT-3Chapter 13. Models that act
  42. 2023LLaMA Turning pointbuilt directly on BPE tokenizationChapter 8. The weights get out
  43. 2023llama.cppwas influenced by LLaMAChapter 9. Making it run anywhere
  44. 2023Alpacabuilt directly on LLaMAChapter 8. The weights get out
  45. 2023Claudebuilt directly on Constitutional AI
    was influenced by HH-RLHF    Chapter 7. The assistant goes public
  46. 2023GPT-4 Turning pointbuilt directly on Scaling lawsChapter 7. The assistant goes public
  47. 2023Reflexionbuilt directly on ReActChapter 13. Models that act
  48. 2023AutoGPTwas influenced by GPT-4Chapter 13. Models that act
  49. 2023Pythiawas influenced by Scaling lawsChapter 8. The weights get out
  50. 2023LLaVAbuilt directly on LLaMAChapter 11. Seeing and hearing
  51. 2023Emergence miragesucceeded Emergent abilitiesChapter 5. Scale becomes a science
  52. 2023100K contextbuilt directly on ClaudeChapter 10. Reading more than you remember
  53. 2023Tree of thoughtsbuilt directly on Chain of thoughtChapter 12. Thinking longer
  54. 2023QLoRAbuilt directly on LLaMAChapter 9. Making it run anywhere
  55. 2023DPObuilt directly on RLHF for summarisation
    built directly on InstructGPT    Chapter 6. Teaching models what people want
  56. 2023Process supervisionbuilt directly on Chain of thoughtChapter 12. Thinking longer
  57. 2023Function callingwas influenced by Toolformer (indirect influence)Chapter 13. Models that act
  58. 2023phi-1was influenced by GPT-3.5
    was influenced by Self-Instruct (indirect influence)    Chapter 8. The weights get out
  59. 2023Position interpolationbuilt directly on LLaMAChapter 10. Reading more than you remember
  60. 2023Lost in the middlewas influenced by 100K context (indirect influence)Chapter 10. Reading more than you remember
  61. 2023Llama 2 Turning pointbuilt directly on LLaMAChapter 8. The weights get out
  62. 2023GPT-4Vwas influenced by CLIP (indirect influence)Chapter 11. Seeing and hearing
  63. 2023SWE-benchwas influenced by GPT-4
    built directly on Codex / HumanEval    Chapter 13. Models that act
  64. 2023Gemini 1.0succeeded PaLMChapter 11. Seeing and hearing
  65. 2023Mixtral 8x7Bbuilt directly on Switch TransformerChapter 8. The weights get out
  66. 2024DeepSeek LLMbuilt directly on Scaling laws
    was influenced by Llama 2    Chapter 8. The weights get out
  67. 2024DeepSeekMoEwas influenced by Switch TransformerChapter 8. The weights get out
  68. 2024Dolmawas influenced by The PileChapter 8. The weights get out
  69. 2024OLMowas influenced by Llama 2Chapter 8. The weights get out
  70. 2024GRPOwas influenced by InstructGPT
    built directly on DeepSeek LLM    Chapter 12. Thinking longer
  71. 2024Gemini 1.5built directly on Gemini 1.0Chapter 10. Reading more than you remember
  72. 2024Gemmawas influenced by Llama 2Chapter 8. The weights get out
  73. 2024Claude 3was influenced by GPT-4Chapter 14. Where the threads meet now
  74. 2024Chatbot Arenawas influenced by ChatGPTChapter 14. Where the threads meet now
  75. 2024Devinwas influenced by GPT-4Chapter 13. Models that act
  76. 2024Llama 3built directly on Llama 2Chapter 8. The weights get out
  77. 2024Phi-3built directly on phi-1Chapter 8. The weights get out
  78. 2024DeepSeek-V2 / MLAbuilt directly on DeepSeekMoEChapter 9. Making it run anywhere
  79. 2024SWE-agentbuilt directly on SWE-bench
    was influenced by ReAct    Chapter 13. Models that act
  80. 2024GPT-4obuilt directly on GPT-4
    built directly on GPT-4V    Chapter 11. Seeing and hearing
  81. 2024Claude 3.5 Sonnetbuilt directly on Claude 3Chapter 14. Where the threads meet now
  82. 2024FineWebwas influenced by The PileChapter 8. The weights get out
  83. 2024Llama 3.1 405Bwas influenced by Position interpolationChapter 8. The weights get out
  84. 2024Test-time computewas influenced by Process supervision
    was influenced by Tree of thoughts (indirect influence)
    was influenced by STaR (indirect influence)    Chapter 12. Thinking longer
  85. 2024o1 Turning pointwas influenced by Chain of thoughtChapter 12. Thinking longer
  86. 2024Computer usewas influenced by GPT-4VChapter 13. Models that act
  87. 2024MCPwas influenced by Toolformer (indirect influence)Chapter 13. Models that act
  88. 2024Gemini 2.0was influenced by Computer use (indirect influence)Chapter 14. Where the threads meet now
  89. 2024o3 announcedbuilt directly on o1Chapter 12. Thinking longer
  90. 2024DeepSeek-V3was influenced by Llama 3.1 405BChapter 8. The weights get out
  91. 2025DeepSeek-R1was influenced by Chain of thoughtChapter 12. Thinking longer
  92. 2025Deep researchwas influenced by WebGPT (indirect influence)Chapter 13. Models that act
  93. 2025Claude 3.7was influenced by o1Chapter 12. Thinking longer
  94. 2025Claude Codewas influenced by Computer use
    was influenced by MCP    Chapter 13. Models that act
  95. 2025GPT-4.5was influenced by Scaling lawsChapter 14. Where the threads meet now
  96. 2025Gemini 2.5was influenced by o1Chapter 12. Thinking longer
  97. 2025Llama 4succeeded Llama 3.1 405B
    was influenced by Switch Transformer    Chapter 8. The weights get out
  98. 2025o3 releasedbuilt directly on o1
    was influenced by Function calling    Chapter 12. Thinking longer
  99. 2025Qwen3was influenced by DeepSeek-R1Chapter 8. The weights get out
  100. 2025Claude 4succeeded Claude 3
    was influenced by SWE-bench    Chapter 13. Models that act
  101. 2025Kimi K2was influenced by DeepSeekMoE
    was influenced by DeepSeek-V3 (indirect influence)
    was influenced by GRPO    Chapter 8. The weights get out
  102. 2025IMO goldbuilt directly on Gemini 2.5
    was influenced by Test-time compute    Chapter 12. Thinking longer
  103. 2025gpt-osswas influenced by DeepSeek-R1 (indirect influence)Chapter 8. The weights get out
  104. 2025GPT-5succeeded GPT-4oChapter 14. Where the threads meet now
  105. 2025GLM-4.5was influenced by DeepSeek-R1 (indirect influence)Chapter 8. The weights get out
  106. 2025DeepSeek-V3.2built directly on DeepSeek-V3Chapter 10. Reading more than you remember
  107. 2025Claude Sonnet 4.5built directly on Claude 4Chapter 13. Models that act
  108. 2025IMO-ProofBenchwas influenced by IMO goldChapter 12. Thinking longer
  109. 2025GPT-5.1built directly on GPT-5Chapter 14. Where the threads meet now
  110. 2025Gemini 3built directly on Gemini 2.5Chapter 14. Where the threads meet now
  111. 2025Claude Opus 4.5was influenced by SWE-benchChapter 13. Models that act
  112. 2025GPT-5.2built directly on GPT-5.1Chapter 14. Where the threads meet now
  113. 2026GPT-5.3-Codexwas influenced by Computer use (indirect influence)Chapter 13. Models that act
  114. 2026Claude Opus 4.6built directly on Claude Opus 4.5Chapter 14. Where the threads meet now
  115. 2026Gemini 3 Deep Thinkbuilt directly on Gemini 3
    was influenced by IMO gold    Chapter 12. Thinking longer
  116. 2026Qwen3.5built directly on Qwen3Chapter 8. The weights get out
  117. 2026Gemini 3.1built directly on Gemini 3Chapter 14. Where the threads meet now
  118. 2026GPT-5.4built directly on GPT-5.3-CodexChapter 14. Where the threads meet now
  119. 2026Project Glasswingwas influenced by Computer use (indirect influence)Chapter 14. Where the threads meet now
  120. 2026Muse Sparksucceeded Llama 4Chapter 8. The weights get out
  121. 2026Claude Opus 4.7built directly on Claude Opus 4.6Chapter 14. Where the threads meet now
  122. 2026GPT-5.5built directly on GPT-5.4Chapter 14. Where the threads meet now
  123. 2026DeepSeek-V4was influenced by DeepSeek-V3Chapter 8. The weights get out
  124. 2026Claude Opus 4.8built directly on Claude Opus 4.7Chapter 14. Where the threads meet now
  125. 2026Claude Sonnet 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  126. 2026Fable 5 / Mythos 5was influenced by Project GlasswingChapter 14. Where the threads meet now
  127. 2026GLM-5.2built directly on GLM-4.5Chapter 8. The weights get out
  128. 2026GPT-5.6 Solbuilt directly on GPT-5.5Chapter 14. Where the threads meet now
  129. 2026Kimi K3built directly on Kimi K2Chapter 8. The weights get out
  130. 2026Claude Opus 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  131. 2026Qwen3.8built directly on Qwen3.5Chapter 8. The weights get out
  132. 2026Claude Opus 5.5built directly on Claude Opus 5Chapter 14. Where the threads meet now

Direct links and their sources

Where it came from

None recorded in the sources used.

What it led to

Built on directly by

Developed alongside

Terms