Chain-of-Thought Prompting Elicits Reasoning in Large Language Models - Summary

Arxiv URL: https://arxiv.org/abs/2201.11903

Authors: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou

Summary:

The paper explores how generating a chain of thought can improve the ability of large language models to perform complex reasoning. The authors introduce a simple method called chain-of-thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain-of-thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks.

Key Insights & Learnings:

• Chain-of-thought prompting significantly improves the ability of large language models to perform complex reasoning.
• Reasoning abilities emerge naturally in sufficiently large language models via chain-of-thought prompting.
• Chain-of-thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks.
• Prompting a PaLM 540B with just eight chain-of-thought exemplars achieves state-of-the-art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.
• Chain-of-thought prompting is a promising approach for facilitating reasoning, providing interpretability, and potentially applicable to any task that humans can solve via language.

Terms Mentioned: Chain-of-thought prompting, Large language models, Arithmetic reasoning, Commonsense reasoning, Symbolic reasoning, Few-shot prompting, Math word problems, PaLM 540B, GSM8K benchmark, GPT-3

Technologies / Libraries Mentioned: Google Research, Neural Information Processing Systems (NeurIPS), arXiv