Low-rank adaptation (LoRA)
A cheap way to fine-tune a large model. Its weights are frozen, and only small extra matrices are trained alongside them, often under 1% of the size of the model.
Introduced by
LoRA, 2021
Used by
- QLoRA, 2023
A cheap way to fine-tune a large model. Its weights are frozen, and only small extra matrices are trained alongside them, often under 1% of the size of the model.
LoRA, 2021