Sparse attention

Attention in which each position looks at only a chosen subset of other positions, instead of all of them. It cuts the cost of attention on long sequences.