Before it
Latent attention had reduced the KV cache, but attention cost still grew with length.
The problem
Long-context inference remained expensive per token as sequences grew.
What it introduced
A trainable sparse attention mechanism that selects which past tokens to attend to, cutting long-context cost sharply.
Why it matters
It revived sparse attention as a frontier technique rather than a compromise, and cut long-context prices.
What modern models inherit
Learned sparsity as the current answer to long-context cost.
What came next
DeepSeek V4's million-token context in 2026.