NLP Lineage

Research thread · 1992 to 2025

Representation and retrieval

22 milestones sit on this thread. For 15 of them it is the main thread. The others also belong to another thread, which is named under them.

1992

  • Class-based n-gramsClass-Based n-gram Models of Natural LanguageMain thread: Language modellingResearch paper

2003

  • Neural LMA Neural Probabilistic Language ModelMain thread: Language modellingResearch paper

2011

  • NLP from scratchNatural Language Processing (Almost) from ScratchMain thread: Language modellingResearch paper

2013

  • word2vecEfficient Estimation of Word Representations in Vector SpaceTechnique

2014

  • GloVeGloVe: Global Vectors for Word RepresentationTechnique

2017

  • FAISSBillion-scale similarity search with GPUsInfrastructure
  • CoVeLearned in Translation: Contextualized Word VectorsMain thread: Language modellingResearch paper

2018

  • ELMoDeep Contextualized Word RepresentationsMain thread: Language modellingModel

2019

  • Sentence-BERTSentence-BERT: Sentence Embeddings using Siamese BERT-NetworksResearch paper

2020

  • Dense retrievalDense Passage Retrieval for Open-Domain Question AnsweringResearch paper
  • RAGRetrieval-Augmented Generation for Knowledge-Intensive NLP TasksMain thread: Language modellingResearch paper

2021

  • CLIPLearning Transferable Visual Models From Natural Language SupervisionModel
  • RETROImproving Language Models by Retrieving from Trillions of TokensMain thread: Language modellingResearch paper
  • WebGPTWebGPT: Browser-assisted question-answering with human feedbackResearch paper

2022

  • ReActReAct: Synergizing Reasoning and Acting in Language ModelsTechnique

2023

  • ToolformerToolformer: Language Models Can Teach Themselves to Use ToolsResearch paper
  • Function callingFunction calling in the OpenAI APIProduct
  • Lost in the middleLost in the Middle: How Language Models Use Long ContextsResearch paper

2024

  • SWE-agentSWE-agent: Agent-Computer Interfaces Enable Automated Software EngineeringResearch paper
  • MCPModel Context ProtocolInfrastructure

2025