The paper discusses eight potentially surprising claims about large language models (LLMs), including their predictable increase in capability with increasing investment, the unpredictability of specific behaviors, and the lack of reliable techniques for steering their behavior.
The paper proposes Automatic Prompt Engineer (APE), an algorithm that generates and selects natural language instructions for large language models (LLMs) to improve task performance. APE treats the instruction as a program and optimizes it by searching over a pool of instruction candidates propose
The paper discusses the limitations of pre-trained language representations in NLP systems and the need for task-specific datasets and fine-tuning. The authors show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with pri