NLP Lineage

15 December 2022 · Alignment

Constitutional AI

Constitutional AI: Harmlessness from AI Feedback

Y. Bai, S. Kadavath, S. Kundu, et al.

Date
15 December 2022
What the date marks
arXiv submission
Kind
Training method
Organisation
Anthropic
Access
Not applicable
Threads
In the guided tour
Chapter 6. Teaching models what people want
Learning phases
2. How LLMs are trained

A milestone: 2 later milestones descend directly from it, and its descendants reach 7 of the 10 threads. How levels are decided

Read the primary source (arxiv.org)

Full summary in progress. Until then, this page shows the milestone's record from the research data.

What it changed

Before it

Harmlessness came from humans labelling large volumes of unpleasant model output.

The problem

That labour is expensive, distressing, and its criteria live in labellers' heads rather than on paper.

What it introduced

Write the principles down, have the model critique and revise its own outputs against them, and train on its own feedback.

Why it matters

It moved part of alignment from tacit labelling to explicit written rules, and made the criteria auditable.

What modern models inherit

AI feedback against stated principles as a supplement to human preference data.

What came next

RLAIF more broadly, and the alignment approach behind Claude.

Before and after

The milestones it descends from, and the milestones that descend from it. Work developed alongside it is not counted. Weak links are left out unless you include them.

Before: where it came from

Constitutional AI has 24 documented ancestors across 7 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 11 turning points among them.

Constitutional AI has 24 documented ancestors across 7 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

Constitutional AI has 33 recorded ancestors, including indirect ones, across 7 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 11 turning points among them.

Constitutional AI has 33 recorded ancestors, including indirect ones, across 7 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

  1. 2022Constitutional AIThis milestone.
  2. 2022HH-RLHFConstitutional AI built directly on thisChapter 6. Teaching models what people want
  3. 2022Chain of thought Turning pointConstitutional AI was influenced by thisChapter 12. Thinking longer
  4. 2021HHH assistantHH-RLHF built directly on thisChapter 6. Teaching models what people want
  5. 2020RLHF for summarisation Turning pointHH-RLHF built directly on thisChapter 6. Teaching models what people want
  6. 2020GPT-3 Turning pointChain of thought was influenced by thisChapter 5. Scale becomes a science
  7. 2020Scaling laws Turning pointGPT-3 was influenced by thisChapter 5. Scale becomes a science
  8. 2019RLHF for LMsRLHF for summarisation built directly on thisChapter 6. Teaching models what people want
  9. 2019Sparse Transformer Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  10. 2019GPT-2 Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  11. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  12. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  13. 2017RLHF originConstitutional AI was influenced by thisChapter 6. Teaching models what people want
  14. 2017Transformer Turning pointGPT-2 built directly on this
    Sparse Transformer built directly on this
    Scaling laws built directly on this    Chapter 3. Attention, then only attention
  15. 2016Layer normGPT-2 built directly on thisChapter 3. Attention, then only attention
  16. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  17. 2015LM pretraining Turning pointGPT-1 was influenced by thisChapter 2. Words become vectors
  18. 2015BPE tokenizationGPT-2 built directly on thisChapter 2. Words become vectors
  19. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  20. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  21. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  22. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  23. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  24. 1997LSTM Turning pointSeq2seq built directly on thisChapter 1. Language as probability
  25. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability
  1. 2022Constitutional AIThis milestone.
  2. 2022HH-RLHFConstitutional AI built directly on thisChapter 6. Teaching models what people want
  3. 2022Chain of thought Turning pointConstitutional AI was influenced by thisChapter 12. Thinking longer
  4. 2021HHH assistantHH-RLHF built directly on thisChapter 6. Teaching models what people want
  5. 2020RLHF for summarisation Turning pointHH-RLHF built directly on thisChapter 6. Teaching models what people want
  6. 2020GPT-3 Turning pointChain of thought was influenced by thisChapter 5. Scale becomes a science
  7. 2020Scaling laws Turning pointGPT-3 was influenced by thisChapter 5. Scale becomes a science
  8. 2019RLHF for LMsRLHF for summarisation built directly on thisChapter 6. Teaching models what people want
  9. 2019Sparse Transformer Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  10. 2019GPT-2 Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  11. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  12. 2019Transformer-XLSparse Transformer was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  13. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  14. 2018ELMoGPT-1 was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  15. 2018ULMFiTGPT-1 was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  16. 2017CoVeELMo was influenced by thisChapter 3. Attention, then only attention
  17. 2017RLHF originConstitutional AI was influenced by thisChapter 6. Teaching models what people want
  18. 2017Transformer Turning pointGPT-2 built directly on this
    Sparse Transformer built directly on this
    Scaling laws built directly on this    Chapter 3. Attention, then only attention
  19. 2017ConvS2STransformer was influenced by this (indirect influence)Chapter 3. Attention, then only attention
  20. 2016Layer normGPT-2 built directly on thisChapter 3. Attention, then only attention
  21. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  22. 2015LM pretraining Turning pointGPT-1 was influenced by thisChapter 2. Words become vectors
  23. 2015BPE tokenizationGPT-2 built directly on thisChapter 2. Words become vectors
  24. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  25. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  26. 2014GloVeELMo built directly on thisChapter 2. Words become vectors
  27. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  28. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  29. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  30. 2013word2vecELMo was influenced by thisChapter 2. Words become vectors
  31. 2011NLP from scratchELMo was influenced by thisChapter 1. Language as probability
  32. 2003Neural LMNLP from scratch was influenced by this
    word2vec was influenced by this    Chapter 1. Language as probability
  33. 1997LSTM Turning pointSeq2seq built directly on this
    ELMo built directly on this    Chapter 1. Language as probability
  34. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability

After: what it led to

Constitutional AI has 16 documented descendants across 5 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing its direct descendants.

Constitutional AI has 16 documented descendants across 5 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing all of them.

Constitutional AI has 31 recorded descendants, including indirect ones, across 5 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing its direct descendants.

Constitutional AI has 31 recorded descendants, including indirect ones, across 5 chapters, reaching forward to Claude Opus 5.5 in 2026. Showing all of them.

  1. 2022Constitutional AIThis milestone.
  2. 2023Claudebuilt directly on Constitutional AIChapter 7. The assistant goes public
  3. 2023100K contextbuilt directly on ClaudeChapter 10. Reading more than you remember
  4. 2024Claude 3built directly on Constitutional AIChapter 14. Where the threads meet now
  5. 2024Claude 3.5 Sonnetbuilt directly on Claude 3Chapter 14. Where the threads meet now
  6. 2024Computer usebuilt directly on Claude 3.5 SonnetChapter 13. Models that act
  7. 2025Claude 3.7built directly on Claude 3.5 SonnetChapter 12. Thinking longer
  8. 2025Claude Codewas influenced by Computer use
    was influenced by Claude 3.7    Chapter 13. Models that act
  9. 2025Claude 4succeeded Claude 3Chapter 13. Models that act
  10. 2025Claude Sonnet 4.5built directly on Claude 4Chapter 13. Models that act
  11. 2025Claude Opus 4.5built directly on Claude Sonnet 4.5Chapter 13. Models that act
  12. 2026Claude Opus 4.6built directly on Claude Opus 4.5Chapter 14. Where the threads meet now
  13. 2026Claude Opus 4.7built directly on Claude Opus 4.6Chapter 14. Where the threads meet now
  14. 2026Claude Opus 4.8built directly on Claude Opus 4.7Chapter 14. Where the threads meet now
  15. 2026Claude Sonnet 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  16. 2026Claude Opus 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  17. 2026Claude Opus 5.5built directly on Claude Opus 5Chapter 14. Where the threads meet now
  1. 2022Constitutional AIThis milestone.
  2. 2023Claudebuilt directly on Constitutional AIChapter 7. The assistant goes public
  3. 2023100K contextbuilt directly on ClaudeChapter 10. Reading more than you remember
  4. 2023Lost in the middlewas influenced by 100K context (indirect influence)Chapter 10. Reading more than you remember
  5. 2024Gemini 1.5was influenced by Lost in the middle (indirect influence)Chapter 10. Reading more than you remember
  6. 2024Claude 3built directly on Constitutional AIChapter 14. Where the threads meet now
  7. 2024Claude 3.5 Sonnetbuilt directly on Claude 3Chapter 14. Where the threads meet now
  8. 2024Computer usebuilt directly on Claude 3.5 SonnetChapter 13. Models that act
  9. 2024Gemini 2.0was influenced by Computer use (indirect influence)Chapter 14. Where the threads meet now
  10. 2025Claude 3.7built directly on Claude 3.5 SonnetChapter 12. Thinking longer
  11. 2025Claude Codewas influenced by Computer use
    was influenced by Claude 3.7    Chapter 13. Models that act
  12. 2025Gemini 2.5built directly on Gemini 2.0
    was influenced by Gemini 1.5    Chapter 12. Thinking longer
  13. 2025Claude 4succeeded Claude 3Chapter 13. Models that act
  14. 2025IMO goldbuilt directly on Gemini 2.5Chapter 12. Thinking longer
  15. 2025Claude Sonnet 4.5built directly on Claude 4Chapter 13. Models that act
  16. 2025IMO-ProofBenchwas influenced by IMO goldChapter 12. Thinking longer
  17. 2025Gemini 3was influenced by Gemini 1.5Chapter 14. Where the threads meet now
  18. 2025Claude Opus 4.5built directly on Claude Sonnet 4.5Chapter 13. Models that act
  19. 2026GPT-5.3-Codexwas influenced by Computer use (indirect influence)Chapter 13. Models that act
  20. 2026Claude Opus 4.6built directly on Claude Opus 4.5Chapter 14. Where the threads meet now
  21. 2026Gemini 3 Deep Thinkbuilt directly on Gemini 3Chapter 12. Thinking longer
  22. 2026Gemini 3.1built directly on Gemini 3Chapter 14. Where the threads meet now
  23. 2026GPT-5.4built directly on GPT-5.3-CodexChapter 14. Where the threads meet now
  24. 2026Project Glasswingwas influenced by Computer use (indirect influence)Chapter 14. Where the threads meet now
  25. 2026Claude Opus 4.7built directly on Claude Opus 4.6Chapter 14. Where the threads meet now
  26. 2026GPT-5.5built directly on GPT-5.4Chapter 14. Where the threads meet now
  27. 2026Claude Opus 4.8built directly on Claude Opus 4.7Chapter 14. Where the threads meet now
  28. 2026Claude Sonnet 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  29. 2026Fable 5 / Mythos 5was influenced by Project GlasswingChapter 14. Where the threads meet now
  30. 2026GPT-5.6 Solbuilt directly on GPT-5.5Chapter 14. Where the threads meet now
  31. 2026Claude Opus 5succeeded Claude Opus 4.8Chapter 14. Where the threads meet now
  32. 2026Claude Opus 5.5built directly on Claude Opus 5Chapter 14. Where the threads meet now

Direct links and their sources

Where it came from

Built directly on

Influenced by

What it led to

Built on directly by