Long short-term memory (LSTM)
A type of recurrent network, which reads a sequence one step at a time. It has a memory cell and gates: small learned switches that decide what to store, keep and output. This lets it carry information across many steps.
Introduced by
LSTM, 1997
Used by
- Seq2seq, 2014
- LM pretraining, 2015
- Sparse MoE, 2017