Drafted with Claude from the paper on 15 October 2026; reviewed by Zeeshan Arif. Read from arxiv.org (arXiv version 6 (January 2023) of the NeurIPS 2022 conference paper).
What it changed
It showed that a large language model solves multi-step problems much better if its prompt examples include the working, not only the answer. The model then writes its own steps before answering. This method, chain-of-thought prompting, needs no training. Later reasoning models build on the same idea: the model writes out its steps before it answers.
The problem before it
GPT-3 had shown few-shot prompting: put a few solved examples in the prompt, and the model does the task. This worked for many simple tasks, but not for tasks that need several steps of reasoning, such as maths word problems. On these, larger models were often not much better, so the scaling curve was flat.
Earlier work had taught models to write out their reasoning steps, by training or fine-tuning on many examples with written solutions. But writing a large set of good solutions is expensive.
The idea
The mechanism. Keep few-shot prompting, but change the examples. Each example is a triple: a question, a chain of thought and the answer. A chain of thought is a short series of reasoning steps in plain language that lead to the answer.
- Standard prompt example: “Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now? The answer is 11.”
- Chain-of-thought example: the same question, then “Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.”
Given such examples, the model writes its own steps for a new question, and then the answer. Nothing is trained: the model is used as it is.
The authors list four reasons this might help:
- The model can spend more steps on harder problems.
- The steps show how the model reached its answer, which helps to find mistakes.
- It can work for any task that people can reason about in language.
- It needs only a few hand-written examples.
The test. The authors wrote 8 example chains of thought for maths word problems and used the same 8 for most maths datasets. They also wrote examples for commonsense tasks and for two simple symbolic tasks. They compared standard prompting with chain-of-thought prompting on several model families and sizes: GPT-3, LaMDA, PaLM, UL2 and Codex.
The notation. There are no equations. A prompt is a list of ⟨input, chain of thought, output⟩ examples, followed by the new input.
What it showed
Reported in the paper. It was published as a peer-reviewed conference paper at NeurIPS 2022. We read version 6 of the arXiv paper. All scores are percentages of problems solved.
-
GSM8K maths word problems. With PaLM 540B, the score rose from 17.9% (standard prompting) to 56.9% (chain of thought). The earlier best was 55%, from a 2021 study that fine-tuned GPT-3 and added a second model to check answers. With an added calculator for the arithmetic, the score was 58.6%.
-
It needs a large model. Chain of thought did not help small models, and it hurt most models below 10 billion parameters: they wrote fluent but illogical steps. The gains appeared at about 100 billion parameters. The authors call it an “emergent ability” of scale.
-
Harder problems gained most. On GSM8K, the score more than doubled for the largest GPT-3 and PaLM models. On the easiest one-step problems, the gain was small or negative.
-
The words matter. Three variations did not work as well:
- writing only the equation before the answer;
- writing a row of dots of the same length, to give extra computation without reasoning;
- writing the reasoning after the answer.
These suggest that the gain comes from the reasoning in plain language, not just from extra text.
-
Robust to who wrote the examples. Examples written by three different authors, and examples taken from the GSM8K training set, all beat standard prompting by a large margin.
-
Commonsense and symbols. With PaLM 540B, it beat the earlier best on StrategyQA (75.6% against 69.4%) and scored 95.4% on a sports-plausibility task, against 84% for an unaided sports fan. On two simple symbolic tasks, it also worked on inputs longer than those in the examples.
-
Reading the steps. In 50 correct answers from LaMDA 137B, almost all chains of thought were also correct. The main text says two were right by chance; appendix D.1 says one. In 50 wrong answers, 46% of the chains of thought were nearly right, with a small error such as a calculation mistake.
Limits, and what came later
The limits come from the paper:
- Is it reasoning? The steps look like human reasoning, but the authors leave open whether the network is “actually reasoning”.
- No guarantee of correct steps. A chain of thought can be wrong, even when the answer is right.
- Only large models. It works only at large scale, which makes it costly to use.
- Prompts still matter. Results varied with who wrote the examples. On a coin-flip task, accuracy ranged from 71.4% to 99.6% between authors.
- Gains vary by model. With GPT-3, chain of thought did not help on the CSQA and StrategyQA commonsense tasks.
- An inconsistency. The abstract and appendices say three language models were tested; section 3.1 lists five.
Much later work built on it. Self-consistency samples many chains of thought and takes the most common answer. Zero-shot chain of thought gets the same effect with no examples. ReAct combines reasoning steps with actions. Tree of Thoughts turns the chain into a search. o1 and DeepSeek-R1 train models to produce long chains of thought, instead of prompting for them.
What to know first
- Few-shot prompting, from GPT-3.
- That model size changes what a model can do (scaling laws).
- What a benchmark like GSM8K is: a fixed set of school maths word problems with known answers.
How to read it
The paper is 9 pages of main text, with 34 more pages of references and appendices, including every prompt used.
- Look at Figure 1. It shows the whole idea in one example.
- Read sections 1 and 2 for the motivation.
- Read section 3 with Figures 4 to 6: results, the ablation study and robustness.
- Skim sections 4 and 5 for commonsense and symbolic tasks.
- Read section 6 for the authors’ limits, and appendix A.2 on prompt engineering.
Expect about 1.5 hours for the steps above (an estimate).
Links
- The paper: arXiv 2201.11903
- Code: the paper does not link code. Appendix G gives the full prompts used.
- Video: none is linked here yet.