Before it
Language was studied as grammar and meaning, not as a measurable signal.
The problem
No way to quantify how predictable a stretch of text is.
What it introduced
Entropy, and the idea that a text source can be modelled as a probability distribution over the next symbol.
Why it matters
Every language model since scores itself on how surprised it is by the next token. That is Shannon's measure.
What modern models inherit
Cross-entropy loss, perplexity, and the framing of language as prediction.
What came next
Statistical speech recognition and machine translation at IBM through the 1970s-80s.