Instruction Tuning with GPT-4 - Summary

Arxiv URL: https://arxiv.org/abs/2304.03277v1

Authors: Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao

Summary:

The paper presents the first attempt to use GPT-4 to generate instruction-following data for Large Language Models (LLMs) finetuning. The 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks compared to the instruction-following data generated by previous state-of-the-art models.

Key Insights & Learnings:

• Finetuning Large Language Models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks.
• Self-Instruct tuning is a simple and effective method of aligning LLMs to human intent, by learning from instruction-following data generated by state-of-the-art instruction-tuned teacher LLMs.
• The recent success of ChatGPT and GPT-4 offers tremendous opportunities to improve open-source LLMs using instruction-tuning.
• The paper presents GPT-4 data, instruction-tuned LLaMA models and reward models, and practical tips of building a general-purpose instruction-following agent powered by LLMs.
• The empirical study validates the effectiveness of using GPT-4-generated data for LLM instruction-tuning and suggests practical tips of building a general-purpose instruction-following agent powered by LLMs.

Terms Mentioned: Large Language Models, LLMs, instruction-tuning, GPT-4, self-instruct tuning, ChatGPT, LLaMA, zero-shot performance, machine-generated instruction-following data, finetuning, natural language instructions, real-world tasks, human-annotated prompts, feedback, public benchmarks, datasets, supervised finetuning, proprietary LLMs, Stanford Alpaca, Vicuna, open-source LLMs, alignment criteria, ROUGE-L, prompt engineering, hyper-parameters, input, output, core dataset, verb-noun pairs

Technologies / Libraries Mentioned: OpenAI