4-bit NormalFloat (NF4)
A way of storing model weights in 4 bits each, with the 16 possible values spread to suit weights that follow a bell-shaped distribution. It lets a large frozen model fit in much less memory during fine-tuning.
Introduced by
QLoRA, 2023