Before it
BERT was good at understanding and GPT at generating, with no clean way to get both.
The problem
Generation tasks needed bidirectional encoding of the input and autoregressive decoding of the output.
What it introduced
Corrupt text in several ways and train an encoder-decoder to reconstruct it.
Why it matters
It became the default seq2seq workhorse, and it is the generator inside the original RAG.
What modern models inherit
Encoder-decoder pretraining for conditional generation.
What came next
Used directly by RAG; the denoising family continued through T5 variants.