1948 to 2026

Timeline

Every milestone by year, under its main thread. Turning points are in bold. Years with no milestones are folded together.

Milestones by year and main thread
Year	Language modelling	Alignment	Representation and retrieval	Efficiency and infrastructure	Reasoning	Open weights	Multimodality	Agents	Products and governance	Scale and data
1948	Information theory
1949 to 1989: no milestones in the data.
1990	Statistical MT Recurrent networks
1991				Mixture of experts
1992	Class-based n-grams
1993 to 1996: no milestones in the data.
1997	LSTM (turning point)
1998 to 2002: no milestones in the data.
2003	Neural LM
2004 to 2009: no milestones in the data.
2010	RNN LM
2011	NLP from scratch
2012: no milestones in the data.
2013			word2vec
2014	GRU / encoder-decoder Attention (turning point) Seq2seq (turning point)		GloVe	Adam
2015	Luong attention BPE tokenization LM pretraining (turning point)			Residual connections
2016	Layer norm GNMT
2017	ConvS2S Transformer (turning point) CoVe	RLHF origin	FAISS	Mixed precision						Sparse MoE (turning point)
2018	ULMFiT ELMo GPT-1 SentencePiece BERT (turning point)
2019	Transformer-XL GPT-2 (turning point) WebText Sparse Transformer (turning point) RoBERTa C4 (turning point) T5 (turning point) BART Multi-query attention	RLHF for LMs	Sentence-BERT							Megatron-LM ZeRO / DeepSpeed
2020	Scaling laws (turning point) Longformer RAG GPT-3 (turning point) BigBird Vision Transformer The Pile	RLHF for summarisation (turning point)	Dense retrieval							GShard (turning point) MMLU
2021	RoPE (turning point) LoRA Codex / HumanEval ALiBi Gopher RETRO	FLAN T0 HHH assistant	CLIP WebGPT					Copilot		Switch Transformer
2022	AlphaCode Chinchilla (turning point) PaLM FlashAttention BLOOM Whisper	LaMDA InstructGPT (turning point) STaR HH-RLHF ChatGPT GPT-3.5 (turning point) Constitutional AI	ReAct	LLM.int8() GPTQ Speculative decoding	Chain of thought (turning point) Self-consistency Zero-shot CoT		Flamingo			MT-NLG OPT-175B Emergent abilities Flan-T5 / Flan-PaLM Self-Instruct
2023	LLaMA (turning point) Pythia Grouped-query attention (turning point) Falcon RefinedWeb phi-1 Llama 2 (turning point) Mistral 7B Mamba Mixtral 8x7B	Alpaca Claude LLaVA DPO	Toolformer Function calling Lost in the middle	llama.cpp 100K context QLoRA AWQ Position interpolation vLLM	Reflexion Tree of thoughts SWE-bench		GPT-4V	AutoGPT		GPT-4 (turning point) Emergence mirage Process supervision Gemini 1.0
2024	DeepSeekMoE OLMo	GRPO Claude 3 o1 (turning point)	SWE-agent MCP	DeepSeek-V2 / MLA FlashAttention-3	o3 announced	Gemma	GPT-4o Claude 3.5 Sonnet Qwen2 Computer use Gemini 2.0	Devin		DeepSeek LLM Dolma Gemini 1.5 Chatbot Arena Llama 3 Phi-3 FineWeb Llama 3.1 405B Test-time compute DeepSeek-V3
2025	DeepSeek-R1 GPT-4.5 Kimi K2 DeepSeek-V3.2	Claude 3.7	Deep research Claude Code	Llama 4 o4-mini	Gemini 2.5 o3 released Qwen3 Claude 4 IMO gold gpt-oss GPT-5 GLM-4.5 Claude Sonnet 4.5 IMO-ProofBench GPT-5.1 Gemini 3 Claude Opus 4.5 GPT-5.2
2026				Claude Sonnet 4.6 DeepSeek-V4	Claude Opus 4.6 Gemini 3 Deep Think Qwen3.5 Gemini 3.1 GPT-5.4 Muse Spark Claude Opus 4.7 GPT-5.5 Claude Opus 4.8 Claude Sonnet 5 GLM-5.2 GPT-5.6 Sol Claude Opus 5 Claude Opus 5.5	Qwen3.8	Kimi K3	GPT-5.3-Codex Project Glasswing Fable 5 / Mythos 5	EU AI Act applies

1948

Information theory Language modelling

1949 to 1989: no milestones in the data.

1990

Statistical MT Language modelling
Recurrent networks Language modelling

1991

Mixture of experts Efficiency and infrastructure

1992

Class-based n-grams Language modelling

1993 to 1996: no milestones in the data.

1997

LSTM (turning point) Language modelling

1998 to 2002: no milestones in the data.

2003

Neural LM Language modelling

2004 to 2009: no milestones in the data.

2010

RNN LM Language modelling

2011

NLP from scratch Language modelling

2012: no milestones in the data.

2013

word2vec Representation and retrieval

2014

GRU / encoder-decoder Language modelling
Attention (turning point) Language modelling
Seq2seq (turning point) Language modelling
GloVe Representation and retrieval
Adam Efficiency and infrastructure

2015

Luong attention Language modelling
BPE tokenization Language modelling
LM pretraining (turning point) Language modelling
Residual connections Efficiency and infrastructure

2016

Layer norm Language modelling
GNMT Language modelling

2017

Sparse MoE (turning point) Scale and data
FAISS Representation and retrieval
ConvS2S Language modelling
RLHF origin Alignment
Transformer (turning point) Language modelling
CoVe Language modelling
Mixed precision Efficiency and infrastructure

2018

ULMFiT Language modelling
ELMo Language modelling
GPT-1 Language modelling
SentencePiece Language modelling
BERT (turning point) Language modelling

2019

Transformer-XL Language modelling
GPT-2 (turning point) Language modelling
WebText Language modelling
Sparse Transformer (turning point) Language modelling
RoBERTa Language modelling
Sentence-BERT Representation and retrieval
Megatron-LM Scale and data
RLHF for LMs Alignment
ZeRO / DeepSpeed Scale and data
C4 (turning point) Language modelling
T5 (turning point) Language modelling
BART Language modelling
Multi-query attention Language modelling

2020

Scaling laws (turning point) Language modelling
Dense retrieval Representation and retrieval
Longformer Language modelling
RAG Language modelling
GPT-3 (turning point) Language modelling
GShard (turning point) Scale and data
BigBird Language modelling
RLHF for summarisation (turning point) Alignment
MMLU Scale and data
Vision Transformer Language modelling
The Pile Language modelling

2021

Switch Transformer Scale and data
CLIP Representation and retrieval
RoPE (turning point) Language modelling
LoRA Language modelling
Copilot Agents
Codex / HumanEval Language modelling
ALiBi Language modelling
FLAN Alignment
T0 Alignment
HHH assistant Alignment
Gopher Language modelling
RETRO Language modelling
WebGPT Representation and retrieval

2022

LaMDA Alignment
Chain of thought (turning point) Reasoning
MT-NLG Scale and data
AlphaCode Language modelling
InstructGPT (turning point) Alignment
Self-consistency Reasoning
STaR Alignment
Chinchilla (turning point) Language modelling
PaLM Language modelling
HH-RLHF Alignment
Flamingo Multimodality
OPT-175B Scale and data
Zero-shot CoT Reasoning
FlashAttention Language modelling
Emergent abilities Scale and data
BLOOM Language modelling
LLM.int8() Efficiency and infrastructure
Whisper Language modelling
ReAct Representation and retrieval
Flan-T5 / Flan-PaLM Scale and data
GPTQ Efficiency and infrastructure
ChatGPT Alignment
GPT-3.5 (turning point) Alignment
Speculative decoding Efficiency and infrastructure
Constitutional AI Alignment
Self-Instruct Scale and data

2023

Toolformer Representation and retrieval
LLaMA (turning point) Language modelling
llama.cpp Efficiency and infrastructure
Alpaca Alignment
Claude Alignment
GPT-4 (turning point) Scale and data
Reflexion Reasoning
AutoGPT Agents
Pythia Language modelling
LLaVA Alignment
Emergence mirage Scale and data
100K context Efficiency and infrastructure
Tree of thoughts Reasoning
Grouped-query attention (turning point) Language modelling
QLoRA Efficiency and infrastructure
Falcon Language modelling
DPO Alignment
Process supervision Scale and data
AWQ Efficiency and infrastructure
RefinedWeb Language modelling
Function calling Representation and retrieval
phi-1 Language modelling
Position interpolation Efficiency and infrastructure
Lost in the middle Representation and retrieval
Llama 2 (turning point) Language modelling
vLLM Efficiency and infrastructure
GPT-4V Multimodality
Mistral 7B Language modelling
SWE-bench Reasoning
Mamba Language modelling
Gemini 1.0 Scale and data
Mixtral 8x7B Language modelling

2024

DeepSeek LLM Scale and data
DeepSeekMoE Language modelling
Dolma Scale and data
OLMo Language modelling
GRPO Alignment
Gemini 1.5 Scale and data
Gemma Open weights
Claude 3 Alignment
Chatbot Arena Scale and data
Devin Agents
Llama 3 Scale and data
Phi-3 Scale and data
DeepSeek-V2 / MLA Efficiency and infrastructure
SWE-agent Representation and retrieval
GPT-4o Multimodality
Claude 3.5 Sonnet Multimodality
FineWeb Scale and data
FlashAttention-3 Efficiency and infrastructure
Qwen2 Multimodality
Llama 3.1 405B Scale and data
Test-time compute Scale and data
o1 (turning point) Alignment
Computer use Multimodality
MCP Representation and retrieval
Gemini 2.0 Multimodality
o3 announced Reasoning
DeepSeek-V3 Scale and data

2025

DeepSeek-R1 Language modelling
Deep research Representation and retrieval
Claude 3.7 Alignment
Claude Code Representation and retrieval
GPT-4.5 Language modelling
Gemini 2.5 Reasoning
Llama 4 Efficiency and infrastructure
o3 released Reasoning
o4-mini Efficiency and infrastructure
Qwen3 Reasoning
Claude 4 Reasoning
Kimi K2 Language modelling
IMO gold Reasoning
gpt-oss Reasoning
GPT-5 Reasoning
GLM-4.5 Reasoning
Claude Sonnet 4.5 Reasoning
DeepSeek-V3.2 Language modelling
IMO-ProofBench Reasoning
GPT-5.1 Reasoning
Gemini 3 Reasoning
Claude Opus 4.5 Reasoning
GPT-5.2 Reasoning

2026

Claude Opus 4.6 Reasoning
GPT-5.3-Codex Agents
Gemini 3 Deep Think Reasoning
Qwen3.5 Reasoning
Claude Sonnet 4.6 Efficiency and infrastructure
Gemini 3.1 Reasoning
GPT-5.4 Reasoning
Project Glasswing Agents
Muse Spark Reasoning
Claude Opus 4.7 Reasoning
GPT-5.5 Reasoning
DeepSeek-V4 Efficiency and infrastructure
Claude Opus 4.8 Reasoning
Claude Sonnet 5 Reasoning
Fable 5 / Mythos 5 Agents
GLM-5.2 Reasoning
GPT-5.6 Sol Reasoning
Kimi K3 Multimodality
Claude Opus 5 Reasoning
EU AI Act applies Products and governance
Qwen3.8 Open weights
Claude Opus 5.5 Reasoning