Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Chain-of-thought prompting shows that step-by-step examples unlock multi-step reasoning in large models — no training changes, just better prompts.
Primary source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (arXiv:2201.11903)
Before the 2022 introduction of Chain-of-Thought prompting, the dominant approach to multi-step reasoning with language models was to scale up parameters and training data. That strategy improved results, but it left a persistent gap: models could solve simple arithmetic or logic puzzles occasionally, yet they failed systematically once a problem required more than one or two intermediate steps. The core insight of the Chain-of-Thought paper was not architectural — it was about how the model was guided to express its own reasoning.
Key takeaways
- Adding intermediate reasoning steps to the prompt improves accuracy across arithmetic, common-sense, and symbolic reasoning tasks.
- The technique requires no parameter updates; it works through careful few-shot example design at inference time.
- Performance gains are most pronounced with larger models, confirming an emergent ability rather than a universal technique.
- The method later spawned an entire family of reasoning-enhancement strategies, from self-consistency to tree-of-thoughts.
What Chain-of-Thought actually changes
In standard few-shot prompting, examples show an input paired with a direct answer. Chain-of-Thought prompting modifies only the examples: each demonstration includes the reasoning steps that lead to the final answer. When the model is then given a new question, it imitates the reasoning format rather than attempting to jump straight to a conclusion.
The change is structural, not cosmetic. By requiring the model to emit intermediate states — numbers, comparisons, logical deductions — the prompt forces the computation to unfold sequentially rather than collapsing into a single predictive step. That sequential structure is what makes arithmetic, multi-hop logic, and common-sense reasoning feasible.
Why it works at scale
The paper demonstrates the effect most clearly on arithmetic benchmarks where earlier models consistently underperformed relative to simpler pattern-matching tasks. With Chain-of-Thought prompts, performance on multi-digit arithmetic, word-problem datasets, and logical deduction tasks improved by large margins — not because the model learned new arithmetic rules, but because the prompt unlocked the model's existing capacity to apply them step by step.
The emergent nature of the effect is critical. Small-scale language models did not show the same improvement, confirming that reasoning is not a universal property of transformer architectures but a capability that activates above certain model sizes. That observation shaped later work on scaling laws, reasoning architectures, and the broader study of emergent abilities.
Limitations and open questions
Chain-of-Thought prompting is not a universal solution. It depends heavily on the quality of the demonstration examples; poorly chosen reasoning steps can mislead rather than guide. The technique also increases token consumption significantly, because every answer includes the intermediate reasoning chain, which raises inference cost and latency. Later methods — including self-consistency decoding — address some of these trade-offs by combining reasoning quality with efficiency.
Another open question is whether reasoning expressed through prompts represents genuine structured cognition or sophisticated imitation of reasoning patterns. The distinction matters for safety and reliability: a model that truly reasons may generalize to novel structures, while one that imitates may break under slight distribution shifts.
Further reading
- Self-Consistency Improves Chain of Thought Reasoning — extends Chain-of-Thought with majority-voting over reasoning paths.
- ReAct: Synergizing Reasoning and Acting in Language Models — combines reasoning with external actions for interactive problem solving.
- Large Language Models are Zero-Shot Reasoners — shows that zero-shot reasoning prompts can sometimes replace few-shot demonstrations.
References
- Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., & Li, Y. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903.
- Wei, J., Wang, X., Schuurmans, D., et al. (2022). Supplementary materials and experimental protocols. arXiv:2201.11903.
Want the next review in your inbox? The briefing collects them — at most one email a week. Read signup data and removal options.
Cite this report
@misc{knowyourml-chain-of-thought-prompting
title = {Chain-of-Thought Prompting Elicits Reasoning in Large Language Models},
author = {{Know Your ML}},
year = {2022},
howpublished = {\url{https://knowyourml.yunusemrevurgun.com/report.php?slug=chain-of-thought-prompting}},
note = {AI-generated review of arXiv:2201.11903. Primary sources are listed on the page.}
}
Machine Learning Decision Toolkit
Turn this review into a decision: the Machine Learning Decision Toolkit packages the evaluation scorecards, workload templates and deployment checks behind model, provider and architecture choices.
References
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (arXiv:2201.11903) — https://arxiv.org/abs/2201.11903
Frequently asked questions
What is Chain-of-Thought prompting?
It is a prompting technique where a language model is shown examples of reasoning before giving an answer, so it learns to emit intermediate steps rather than jumping to conclusions.
Does Chain-of-Thought work for all models?
It works best with larger models; small models may still fail even with the prompt because they lack the capacity to maintain coherent intermediate reasoning steps.
Part of
Beyond the paper
How to Remain Valuable When Intelligence Becomes Cheap
This report covered one more way the machine gets smarter. The companion eBook asks the question on the other side: which human advantages stay scarce — and valuable — when intelligence itself is cheap?