Before it
Training used SGD with hand-tuned learning-rate schedules per task.
The problem
Learning rates that suit one parameter suit another badly, and tuning was fragile.
What it introduced
Per-parameter adaptive step sizes from running estimates of gradient mean and variance.
Why it matters
Almost every transformer since has been trained with Adam or a close variant. It removed a whole class of tuning failure.
What modern models inherit
The default optimizer of the field.
What came next
AdamW, and the optimizer-state sharding that ZeRO later required.