Sign in Subscribe

Paper Summaries

LoRA: Low-Rank Adaptation of Large Language Models - Summary

The paper proposes Low-Rank Adaptation (LoRA) as an approach to reduce the number of trainable parameters for downstream tasks in natural language processing. LoRA injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models - Summary

The paper explores how generating a chain of thought can improve the ability of large language models to perform complex reasoning. The authors introduce a simple method called chain-of-thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiment

Language Models are Few-Shot Learners - Summary

The paper discusses the limitations of pre-trained language representations in NLP systems and the need for task-specific datasets and fine-tuning. The authors show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with pri

Attention Is All You Need - Summary

The paper proposes a new network architecture called Transformer that relies solely on attention mechanisms, dispensing with recurrence and convolutions entirely. The Transformer allows for significantly more parallelization and can reach a new state of the art in translation quality while requirin

CAMEL: Communicative Agents for "Mind" Exploration of LLMs - Summary

The paper proposes a novel communicative agent framework named role-playing to facilitate autonomous cooperation among communicative agents and provide insight into their “cognitive” processes. The approach involves using inception prompting to guide chat agents toward task completion while maintai