Eight Things to Know about Large Language Models - Summary
The paper discusses eight potentially surprising claims about large language models (LLMs), including their predictable increase in capability with increasing investment, the unpredictability of specific behaviors, and the lack of reliable techniques for steering their behavior.
Arxiv URL: https://arxiv.org/abs/2304.00612v1
Authors: Samuel R. Bowman
Summary:
The paper discusses eight potentially surprising claims about large language models (LLMs), including their predictable increase in capability with increasing investment, the unpredictability of specific behaviors, and the lack of reliable techniques for steering their behavior.
Key Insights & Learnings:
- LLMs become more capable with increasing investment, even without targeted improvements.
- Specific important behaviors in LLMs tend to emerge unpredictably as a byproduct of increasing investment.
- LLMs often learn and use representations of the outside world.
- There are no reliable techniques for steering the behavior of LLMs.
- Human performance on a task isn't an upper bound on LLM performance.
Terms Mentioned: large language models, scaling laws, pretraining test loss, few-shot learning, chain-of-thought reasoning
Technologies / Libraries Mentioned: OpenAI
