NLP Lineage

Glossary

Mixed precision training

Training with 16-bit numbers for most calculations, which is faster and uses less memory, while keeping a 32-bit copy of the weights so that small updates are not lost.

Introduced by

Mixed precision, 2017

Used by