Before it
Retrieval was bolted on at inference time to a model trained without it.
The problem
If a model can look things up, it should not have to memorise them in its weights.
What it introduced
Retrieve from a two-trillion-token database during pretraining itself, via chunked cross-attention.
Why it matters
It showed a 7B model with retrieval matching much larger ones, arguing memory and reasoning can be separated.
What modern models inherit
The open question of what belongs in weights versus in an external store.
What came next
The idea recurs in long-context and memory work, though frontier labs largely chose bigger contexts instead.