Chain-of-Thought Prompting Elicits Reasoning in Large Language Models - Summary
The paper explores how generating a chain of thought can improve the ability of large language models to perform complex reasoning. The authors introduce a simple method called chain-of-thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiment
Arxiv URL: https://arxiv.org/abs/2201.11903
Authors: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou
Summary:
The paper explores how generating a chain of thought can improve the ability of large language models to perform complex reasoning. The authors introduce a simple method called chain-of-thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain-of-thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks.
Key Insights & Learnings:
- Chain-of-thought prompting significantly improves the ability of large language models to perform complex reasoning.
- Reasoning abilities emerge naturally in sufficiently large language models via chain-of-thought prompting.
- Chain-of-thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks.
- Prompting a PaLM 540B with just eight chain-of-thought exemplars achieves state-of-the-art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.
- Chain-of-thought prompting is a promising approach for facilitating reasoning, providing interpretability, and potentially applicable to any task that humans can solve via language.
Terms Mentioned: Chain-of-thought prompting, Large language models, Arithmetic reasoning, Commonsense reasoning, Symbolic reasoning, Few-shot prompting, Math word problems, PaLM 540B, GSM8K benchmark, GPT-3
Technologies / Libraries Mentioned: Google Research, Neural Information Processing Systems (NeurIPS), arXiv