NLP Lineage

20 March 2023 · Reasoning

Reflexion

Reflexion: Language Agents with Verbal Reinforcement Learning

N. Shinn, F. Cassano, E. Berman, A. Gopinath, K. Narasimhan, S. Yao

Date
20 March 2023
What the date marks
arXiv submission
Kind
Research paper
Organisation
Northeastern / MIT / Princeton
Access
MIT
Other dates
NeurIPS 2023
Threads
In the guided tour
Chapter 13. Models that act
Learning phases
7. Agents

Notable: no later milestone in the data is documented as descending from it. How levels are decided

Full summary in progress. Until then, this page shows the milestone's record from the research data.

What it changed

Before it

An agent that failed a task simply failed and started over.

The problem

Improving from failure normally means gradient updates, which is impossible at inference time.

What it introduced

Have the agent write a verbal post-mortem of its failure and keep it in memory for the next attempt.

Why it matters

Learning from experience without touching the weights, which became a standard pattern in agent design.

What modern models inherit

Episodic self-reflection as an agent's cheap substitute for training.

What came next

Memory and self-critique layers in agent frameworks and coding agents.

Before and after

The milestones it descends from, and the milestones that descend from it. Work developed alongside it is not counted. Weak links are left out unless you include them.

Before: where it came from

Reflexion has 24 documented ancestors across 8 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 11 turning points among them.

Reflexion has 24 documented ancestors across 8 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

Reflexion has 33 recorded ancestors, including indirect ones, across 8 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 11 turning points among them.

Reflexion has 33 recorded ancestors, including indirect ones, across 8 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

  1. 2023ReflexionThis milestone.
  2. 2022ReActReflexion built directly on thisChapter 13. Models that act
  3. 2022Chain of thought Turning pointReAct built directly on thisChapter 12. Thinking longer
  4. 2021WebGPTReAct was influenced by thisChapter 13. Models that act
  5. 2020RLHF for summarisation Turning pointWebGPT built directly on thisChapter 6. Teaching models what people want
  6. 2020GPT-3 Turning pointChain of thought was influenced by this
    WebGPT built directly on this
    Chapter 5. Scale becomes a science
  7. 2020Scaling laws Turning pointGPT-3 was influenced by thisChapter 5. Scale becomes a science
  8. 2019RLHF for LMsRLHF for summarisation built directly on thisChapter 6. Teaching models what people want
  9. 2019Sparse Transformer Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  10. 2019GPT-2 Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  11. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  12. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  13. 2017RLHF originRLHF for summarisation built directly on thisChapter 6. Teaching models what people want
  14. 2017Transformer Turning pointGPT-2 built directly on this
    Sparse Transformer built directly on this
    Scaling laws built directly on this
    Chapter 3. Attention, then only attention
  15. 2016Layer normGPT-2 built directly on thisChapter 3. Attention, then only attention
  16. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  17. 2015LM pretraining Turning pointGPT-1 was influenced by thisChapter 2. Words become vectors
  18. 2015BPE tokenizationGPT-2 built directly on thisChapter 2. Words become vectors
  19. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  20. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  21. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  22. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  23. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  24. 1997LSTM Turning pointSeq2seq built directly on thisChapter 1. Language as probability
  25. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability
  1. 2023ReflexionThis milestone.
  2. 2022ReActReflexion built directly on thisChapter 13. Models that act
  3. 2022Chain of thought Turning pointReAct built directly on thisChapter 12. Thinking longer
  4. 2021WebGPTReAct was influenced by thisChapter 13. Models that act
  5. 2020RLHF for summarisation Turning pointWebGPT built directly on thisChapter 6. Teaching models what people want
  6. 2020GPT-3 Turning pointChain of thought was influenced by this
    WebGPT built directly on this
    Chapter 5. Scale becomes a science
  7. 2020Scaling laws Turning pointGPT-3 was influenced by thisChapter 5. Scale becomes a science
  8. 2019RLHF for LMsRLHF for summarisation built directly on thisChapter 6. Teaching models what people want
  9. 2019Sparse Transformer Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  10. 2019GPT-2 Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  11. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  12. 2019Transformer-XLSparse Transformer was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  13. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  14. 2018ELMoGPT-1 was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  15. 2018ULMFiTGPT-1 was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  16. 2017CoVeELMo was influenced by thisChapter 3. Attention, then only attention
  17. 2017RLHF originRLHF for summarisation built directly on thisChapter 6. Teaching models what people want
  18. 2017Transformer Turning pointGPT-2 built directly on this
    Sparse Transformer built directly on this
    Scaling laws built directly on this
    Chapter 3. Attention, then only attention
  19. 2017ConvS2STransformer was influenced by this (indirect influence)Chapter 3. Attention, then only attention
  20. 2016Layer normGPT-2 built directly on thisChapter 3. Attention, then only attention
  21. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  22. 2015LM pretraining Turning pointGPT-1 was influenced by thisChapter 2. Words become vectors
  23. 2015BPE tokenizationGPT-2 built directly on thisChapter 2. Words become vectors
  24. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  25. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  26. 2014GloVeELMo built directly on thisChapter 2. Words become vectors
  27. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  28. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  29. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  30. 2013word2vecELMo was influenced by thisChapter 2. Words become vectors
  31. 2011NLP from scratchELMo was influenced by thisChapter 1. Language as probability
  32. 2003Neural LMNLP from scratch was influenced by this
    word2vec was influenced by this
    Chapter 1. Language as probability
  33. 1997LSTM Turning pointSeq2seq built directly on this
    ELMo built directly on this
    Chapter 1. Language as probability
  34. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability

After: what it led to

No documented descendants are recorded for Reflexion without weak links.

No documented descendants are recorded for Reflexion without weak links.

No recorded descendants are recorded for Reflexion.

No recorded descendants are recorded for Reflexion.

Direct links and their sources

Where it came from

Built directly on

What it led to

None recorded in the sources used.