Levels
How levels are decided
Each milestone has one of three levels. The level is measured from the documented relationships in the data. Nobody chooses it by hand, and a check rejects any level that does not follow the rule.
The rule
- Turning point (25)
- At least 4 later milestones build on it directly, were influenced by it, or succeeded it. Its descendants also reach at least 5 of the 10 research threads.
- Milestone (131)
- At least one later milestone descends from it, but not enough for a turning point. Recent work with no successors yet is also here.
- Notable (44)
- No later milestone in the data is documented as descending from it, and it is dated before 2025.
Only firm links count: links whose evidence is documented or strong. Weak links (indirect influence, timing only, or a later reading) do not count. Work developed alongside a milestone is not counted as descending from it. These are the same rules as "Before and after" on each milestone page.
What the rule cannot see
- It measures documented influence, not fame. A famous milestone can be a "milestone" when few later milestones in the data are documented as building on it directly. ChatGPT and word2vec are examples. The fix is to document more relationships, with sources, not to change the level by hand.
- Recent work cannot qualify yet. A milestone from 2025 or later has had little time to gain successors, so it is never called notable, and its page says it is too recent to judge.
- The data has 200 milestones. Influence on work outside the data is not counted.
The 25 turning points
- LSTMLong Short-Term Memory1997. 6 direct, 10 threads
- AttentionNeural Machine Translation by Jointly Learning to Align and Translate2014. 4 direct, 10 threads
- Seq2seqSequence to Sequence Learning with Neural Networks2014. 4 direct, 10 threads
- LM pretrainingSemi-supervised Sequence Learning2015. 4 direct, 10 threads
- Sparse MoEOutrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer2017. 4 direct, 9 threads
- TransformerAttention Is All You Need2017. 18 direct, 10 threads
- BERTBERT: Pre-training of Deep Bidirectional Transformers2018. 6 direct, 10 threads
- GPT-2Language Models are Unsupervised Multitask Learners2019. 6 direct, 10 threads
- Sparse TransformerGenerating Long Sequences with Sparse Transformers2019. 4 direct, 10 threads
- C4C4: Colossal Clean Crawled Corpus2019. 4 direct, 10 threads
- T5Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019. 4 direct, 10 threads
- Scaling lawsScaling Laws for Neural Language Models2020. 8 direct, 10 threads
- GPT-3Language Models are Few-Shot Learners2020. 13 direct, 10 threads
- GShardGShard: Scaling Giant Models with Conditional Computation2020. 6 direct, 9 threads
- RLHF for summarisationLearning to Summarize from Human Feedback2020. 5 direct, 10 threads
- RoPERoFormer: Enhanced Transformer with Rotary Position Embedding2021. 4 direct, 9 threads
- Chain of thoughtChain-of-Thought Prompting Elicits Reasoning in Large Language Models2022. 12 direct, 10 threads
- InstructGPTTraining language models to follow instructions with human feedback2022. 6 direct, 10 threads
- ChinchillaTraining Compute-Optimal Large Language Models2022. 4 direct, 9 threads
- GPT-3.5GPT-3.5 series2022. 4 direct, 10 threads
- LLaMALLaMA: Open and Efficient Foundation Language Models2023. 6 direct, 9 threads
- GPT-4GPT-4 Technical Report2023. 7 direct, 10 threads
- Grouped-query attentionGQA: Training Generalized Multi-Query Transformer Models2023. 6 direct, 9 threads
- Llama 2Llama 2: Open Foundation and Fine-Tuned Chat Models2023. 4 direct, 9 threads
- o1OpenAI o12024. 5 direct, 8 threads