5 steps · about 45 min · 2022–2023

Teaching models to reason

Prompting as a reasoning interface: chain-of-thought, self-consistency, tool-using ReAct agents and deliberate search over thoughts.

Fig. 02 / Reading route5 papers, in order
Reading path: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models; Self-Consistency Improves Chain of Thought Reasoning in Language Models; ReAct: Reasoning and Acting; Tree of Thoughts; Causal Attention for Interpretable Multi-Step Reasoning0102030405
Start with Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Follow the numbered steps to Causal Attention for Interpretable Multi-Step Reasoning.
  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models 2022 · 9 min read · Beginner — Chain-of-thought prompting shows that step-by-step examples unlock multi-step reasoning in large models — no training changes, just better prompts.
  2. Self-Consistency Improves Chain of Thought Reasoning in Language Models 2022 · 9 min read · Intermediate — An AI-generated review of Self-Consistency decoding, showing how sampling multiple reasoning paths and selecting the most consistent answer improves.
  3. ReAct: Reasoning and Acting 2022 · 5 min read · Beginner — ReAct interleaves chain-of-thought reasoning with tool actions like search and lookup, grounding language models in external feedback.
  4. Tree of Thoughts 2023 · 5 min read · Intermediate — Tree of Thoughts explores multiple reasoning branches with search and self-evaluation, solving problems where single-pass chain-of-thought fails.
  5. Causal Attention for Interpretable Multi-Step Reasoning 2023 · 17 min read · Advanced — A synthetic proposal for causal attention: separating correlational co-occurrence from causal influence so transformers can reason over interventions.

Every step as a full report

Prefer to jump around? Each step is a standalone report with its own citations, figures and FAQ.

ReAct: Reasoning and Acting

Applications & prompting · 5 min read · Beginner

ReAct interleaves chain-of-thought reasoning with tool actions like search and lookup, grounding language models in external feedback.

reactagentstool-use Read

Tree of Thoughts

Applications & prompting · 5 min read · Intermediate

Tree of Thoughts explores multiple reasoning branches with search and self-evaluation, solving problems where single-pass chain-of-thought fails.

tree-of-thoughtsreasoningsearch Read

Causal Attention for Interpretable Multi-Step Reasoning

Architectures · 17 min read · Advanced

A synthetic proposal for causal attention: separating correlational co-occurrence from causal influence so transformers can reason over interventions.

causal-inferenceattentioninterpretability Read