Instruction tuning
Fine-tuning a language model on many different tasks, each written as a plain-language instruction with its answer, so that the model learns to follow instructions for tasks it has not seen.
Introduced by
FLAN, 2021
Used by
- Flan-T5 / Flan-PaLM, 2022
- Self-Instruct, 2022
- LLaMA, 2023
- Alpaca, 2023
- QLoRA, 2023