NLP Lineage

10 March 2023 · Efficiency and infrastructure

llama.cpp

llama.cpp

Georgi Gerganov and contributors

Date
10 March 2023
What the date marks
Infrastructure
Kind
Infrastructure
Organisation
community
Access
MIT
Other dates
GGUF format 2023-08
Threads
In the guided tour
Chapter 9. Making it run anywhere
Learning phases
6. Local models and deployment

Notable: no later milestone in the data is documented as descending from it. How levels are decided

Read the primary source (github.com)

Full summary in progress. Until then, this page shows the milestone's record from the research data.

What it changed

Before it

Running a model meant Python, CUDA and a datacentre GPU.

The problem

Most people have a laptop, not an A100.

What it introduced

A dependency-free C++ inference engine with quantized weights, running LLaMA on a CPU or a MacBook.

Why it matters

It created local inference as a mass activity, and GGUF became the format the whole local ecosystem shares.

What modern models inherit

The assumption that you can run a capable model on your own machine.

What came next

Ollama and the local-model tooling built on top of it.

Before and after

The milestones it descends from, and the milestones that descend from it. Work developed alongside it is not counted. Weak links are left out unless you include them.

Before: where it came from

llama.cpp has 21 documented ancestors across 7 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 10 turning points among them.

llama.cpp has 21 documented ancestors across 7 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

llama.cpp has 39 recorded ancestors, including indirect ones, across 8 chapters, reaching back to Recurrent networks in 1990. Showing its direct parents and the 14 turning points among them.

llama.cpp has 39 recorded ancestors, including indirect ones, across 8 chapters, reaching back to Recurrent networks in 1990. Showing all of them.

  1. 2023llama.cppThis milestone.
  2. 2023LLaMA Turning pointllama.cpp was influenced by thisChapter 8. The weights get out
  3. 2022Chinchilla Turning pointLLaMA built directly on thisChapter 5. Scale becomes a science
  4. 2021GopherChinchilla built directly on thisChapter 5. Scale becomes a science
  5. 2021RoPE Turning pointLLaMA built directly on thisChapter 10. Reading more than you remember
  6. 2020Scaling laws Turning pointChinchilla built directly on thisChapter 5. Scale becomes a science
  7. 2019GPT-2 Turning pointScaling laws was influenced by thisChapter 4. Pretrain once, use everywhere
  8. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  9. 2019Transformer-XLRoPE was influenced by thisChapter 4. Pretrain once, use everywhere
  10. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  11. 2017Transformer Turning pointRoPE built directly on thisChapter 3. Attention, then only attention
  12. 2016Layer normTransformer built directly on thisChapter 3. Attention, then only attention
  13. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  14. 2015LM pretraining Turning pointGPT-1 was influenced by thisChapter 2. Words become vectors
  15. 2015BPE tokenizationLLaMA built directly on thisChapter 2. Words become vectors
  16. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  17. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  18. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  19. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  20. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  21. 1997LSTM Turning pointSeq2seq built directly on thisChapter 1. Language as probability
  22. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability
  1. 2023llama.cppThis milestone.
  2. 2023LLaMA Turning pointllama.cpp was influenced by thisChapter 8. The weights get out
  3. 2022GPTQllama.cpp was influenced by this (indirect influence)Chapter 9. Making it run anywhere
  4. 2022LLM.int8()GPTQ was influenced by thisChapter 9. Making it run anywhere
  5. 2022OPT-175BLLM.int8() was influenced by thisChapter 8. The weights get out
  6. 2022Chinchilla Turning pointLLaMA built directly on thisChapter 5. Scale becomes a science
  7. 2021GopherChinchilla built directly on thisChapter 5. Scale becomes a science
  8. 2021RoPE Turning pointLLaMA built directly on thisChapter 10. Reading more than you remember
  9. 2020The PileOPT-175B built directly on thisChapter 5. Scale becomes a science
  10. 2020GPT-3 Turning pointOPT-175B built directly on thisChapter 5. Scale becomes a science
  11. 2020Scaling laws Turning pointChinchilla built directly on thisChapter 5. Scale becomes a science
  12. 2019C4 Turning pointThe Pile was influenced by thisChapter 4. Pretrain once, use everywhere
  13. 2019Megatron-LMOPT-175B was influenced by thisChapter 4. Pretrain once, use everywhere
  14. 2019Sparse Transformer Turning pointGPT-3 built directly on thisChapter 4. Pretrain once, use everywhere
  15. 2019GPT-2 Turning pointScaling laws was influenced by thisChapter 4. Pretrain once, use everywhere
  16. 2019WebTextGPT-2 was influenced by thisChapter 4. Pretrain once, use everywhere
  17. 2019Transformer-XLRoPE was influenced by thisChapter 4. Pretrain once, use everywhere
  18. 2018BERT Turning pointMegatron-LM was influenced by thisChapter 4. Pretrain once, use everywhere
  19. 2018GPT-1GPT-2 built directly on thisChapter 4. Pretrain once, use everywhere
  20. 2018ELMoBERT was influenced by this
    GPT-1 was influenced by this (indirect influence)    Chapter 4. Pretrain once, use everywhere
  21. 2018ULMFiTBERT was influenced by this
    GPT-1 was influenced by this (indirect influence)    Chapter 4. Pretrain once, use everywhere
  22. 2017Mixed precisionMegatron-LM was influenced by thisChapter 9. Making it run anywhere
  23. 2017CoVeELMo was influenced by thisChapter 3. Attention, then only attention
  24. 2017Transformer Turning pointRoPE built directly on thisChapter 3. Attention, then only attention
  25. 2017ConvS2STransformer was influenced by this (indirect influence)Chapter 3. Attention, then only attention
  26. 2016Layer normTransformer built directly on thisChapter 3. Attention, then only attention
  27. 2015Residual connectionsTransformer was influenced by thisChapter 3. Attention, then only attention
  28. 2015LM pretraining Turning pointBERT was influenced by this
    GPT-1 was influenced by this    Chapter 2. Words become vectors
  29. 2015BPE tokenizationLLaMA built directly on thisChapter 2. Words become vectors
  30. 2015Luong attentionTransformer was influenced by thisChapter 3. Attention, then only attention
  31. 2014AdamTransformer was influenced by thisChapter 2. Words become vectors
  32. 2014GloVeELMo built directly on thisChapter 2. Words become vectors
  33. 2014Seq2seq Turning pointTransformer built directly on thisChapter 2. Words become vectors
  34. 2014Attention Turning pointTransformer built directly on thisChapter 3. Attention, then only attention
  35. 2014GRU / encoder-decoderAttention built directly on thisChapter 2. Words become vectors
  36. 2013word2vecELMo was influenced by thisChapter 2. Words become vectors
  37. 2011NLP from scratchELMo was influenced by thisChapter 1. Language as probability
  38. 2003Neural LMNLP from scratch was influenced by this
    word2vec was influenced by this    Chapter 1. Language as probability
  39. 1997LSTM Turning pointSeq2seq built directly on thisChapter 1. Language as probability
  40. 1990Recurrent networksLSTM was influenced by thisChapter 1. Language as probability

After: what it led to

No documented descendants are recorded for llama.cpp without weak links.

No documented descendants are recorded for llama.cpp without weak links.

No recorded descendants are recorded for llama.cpp.

No recorded descendants are recorded for llama.cpp.

Direct links and their sources

Where it came from

Influenced by

What it led to

None recorded in the sources used.