PaperTechnology
Attention Is All You Needby Ashish Vaswani et al.
Learn the paper's main ideas in 3 minutes, then hold your own in any conversation about it.
201715 pages✦ Big ideas · 3 min
Rate it
Learn the paper's main ideas in 3 minutes, then hold your own in any conversation about it.

Big ideas · from the full text
4 illustrated ideas you will actually remember, drawn from the complete paper.
The 2017 paper from eight Google researchers that introduced the transformer: a model that reads a whole sentence at once and learns which words to pay attention to, instead of plodding through it one word at a time. Every large language model since, including the GPT family, is built on it. Free to read on arXiv at arxiv.org/abs/1706.03762.
Public statements we checked against the original source
“a famous Google paper in 2017 called Attention is All You Need”
Portraits from Wikimedia Commons · photo credits and licences