Interactive Timeline Explainer

The Attention Timeline

How Transformers Learned to Pay Less

Vanilla attention wasn't wrong — it was expensive. Every mechanism that followed is someone looking at that bill and trying to pay less of it. Laid out chronologically, you can watch the field change its mind: first it wants exactness, then memory, then length, then memory again.

Scroll to explore

Compare Mechanisms

About This Project

An interactive, chronological visual guide to modern attention mechanisms. Every date has been verified against the original paper or release.

The Core Insight

"Every technique substitutes structure for stored parameters — buying generalization at the cost of expressiveness."

Sources

All dates verified against arXiv submissions and official releases. See each mechanism card for specific paper references.

© 2026 · Attention Mechanisms Timeline · Collated by Sunil Jakkaraju