NLP Lineage

29 June 2021 · Agents

Copilot

GitHub Copilot

GitHub

Date
29 June 2021
What the date marks
Product release
Kind
Product
Organisation
GitHub / OpenAI
Access
subscription
Other dates
GA June 2022
Threads
In the guided tour
Chapter 7. The assistant goes public
Learning phases
3. API usage and prompting

Notable: no later milestone in the data is documented as descending from it. How levels are decided

Read the primary source (github.blog)

Full summary in progress. Until then, this page shows the milestone's record from the research data.

What it changed

Before it

Code completion used static analysis of the current file and project.

The problem

Nobody had shipped a large language model into a daily professional workflow.

What it introduced

Codex behind an editor plugin, suggesting whole functions from a comment or signature.

Why it matters

The first mass-market LLM product, eighteen months before ChatGPT, and the start of AI coding as a category.

What modern models inherit

AI assistance as a normal part of writing software.

What came next

Chat interfaces, then agentic coding tools that edit repositories directly.

Before and after

The milestones it descends from, and the milestones that descend from it. Work developed alongside it is not counted. Weak links are left out unless you include them.

Before: where it came from

Copilot has 19 documented ancestors across 6 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 9 turning points among them.

Copilot has 19 documented ancestors across 6 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

Copilot has 28 recorded ancestors, including indirect ones, across 6 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 9 turning points among them.

Copilot has 28 recorded ancestors, including indirect ones, across 6 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

  1. 2021CopilotThis milestone.
  2. 2021Codex / HumanEvalCopilot was influenced by thisChapter 13. Models that act
  3. 2020GPT-3 Turning pointCodex / HumanEval built directly on thisChapter 5. Scale becomes a science
  4. 2020Scaling laws Turning pointGPT-3 was influenced by thisChapter 5. Scale becomes a science
  5. 2019Sparse Transformer Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  6. 2019GPT-2 Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  7. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  8. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  9. 2017Transformer Turning pointGPT-2 built directly on this
    Sparse Transformer built directly on this
    Scaling laws built directly on this    Chapter 3. Attention, then only attention
  10. 2016Layer normGPT-2 built directly on thisChapter 3. Attention, then only attention
  11. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  12. 2015LM pretraining Turning pointGPT-1 was influenced by thisChapter 2. Words become vectors
  13. 2015BPE tokenizationGPT-2 built directly on thisChapter 2. Words become vectors
  14. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  15. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  16. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  17. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  18. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  19. 1997LSTM Turning pointSeq2seq built directly on thisChapter 1. Language as probability
  20. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability
  1. 2021CopilotThis milestone.
  2. 2021Codex / HumanEvalCopilot was influenced by thisChapter 13. Models that act
  3. 2020GPT-3 Turning pointCodex / HumanEval built directly on thisChapter 5. Scale becomes a science
  4. 2020Scaling laws Turning pointGPT-3 was influenced by thisChapter 5. Scale becomes a science
  5. 2019Sparse Transformer Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  6. 2019GPT-2 Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  7. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  8. 2019Transformer-XLSparse Transformer was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  9. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  10. 2018ELMoGPT-1 was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  11. 2018ULMFiTGPT-1 was influenced by this (indirect influence)Chapter 4. Pretrain once, use everywhere
  12. 2017CoVeELMo was influenced by thisChapter 3. Attention, then only attention
  13. 2017Transformer Turning pointGPT-2 built directly on this
    Sparse Transformer built directly on this
    Scaling laws built directly on this    Chapter 3. Attention, then only attention
  14. 2017ConvS2STransformer was influenced by this (indirect influence)Chapter 3. Attention, then only attention
  15. 2016Layer normGPT-2 built directly on thisChapter 3. Attention, then only attention
  16. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  17. 2015LM pretraining Turning pointGPT-1 was influenced by thisChapter 2. Words become vectors
  18. 2015BPE tokenizationGPT-2 built directly on thisChapter 2. Words become vectors
  19. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  20. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  21. 2014GloVeELMo built directly on thisChapter 2. Words become vectors
  22. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  23. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  24. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  25. 2013word2vecELMo was influenced by thisChapter 2. Words become vectors
  26. 2011NLP from scratchELMo was influenced by thisChapter 1. Language as probability
  27. 2003Neural LMNLP from scratch was influenced by this
    word2vec was influenced by this    Chapter 1. Language as probability
  28. 1997LSTM Turning pointSeq2seq built directly on this
    ELMo built directly on this    Chapter 1. Language as probability
  29. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability

After: what it led to

No documented descendants are recorded for Copilot without weak links.

No documented descendants are recorded for Copilot without weak links.

No recorded descendants are recorded for Copilot.

No recorded descendants are recorded for Copilot.

Direct links and their sources

Where it came from

Influenced by

What it led to

None recorded in the sources used.