Attention Is All You Need by Ashish Vaswani et al. — book cover
PaperTechnology

Attention Is All You Needby Ashish Vaswani et al.

Learn the paper's main ideas in 3 minutes, then hold your own in any conversation about it.

201715 pages✦ Big ideas · 3 min
Rate it
A central hub node connected by many lines to surrounding nodes arranged in a ring, drawn in a simple conceptual style.

Big ideas · from the full text

Get smarter about Attention Is All You Need in 3 minutes

4 illustrated ideas you will actually remember, drawn from the complete paper.

  1. 1Attention becomes the center of the model
  2. 2The design must add order and separate attention patterns
  3. 3Parallel training helps, but the cost does not vanish
  4. 4The experiments support the claim within a clear scope
✦ Get smarter in 3 min→
What is Attention Is All You Need about?

The 2017 paper from eight Google researchers that introduced the transformer: a model that reads a whole sentence at once and learns which words to pay attention to, instead of plodding through it one word at a time. Every large language model since, including the GPT family, is built on it. Free to read on arXiv at arxiv.org/abs/1706.03762.

Most readers stop hereGet smarter about Attention Is All You Need19 minutes: the ideas, the context, the criticism, and what to say about itMake me smarter

Who's talking about Attention Is All You Need

Public statements we checked against the original source

Portraits from Wikimedia Commons · photo credits and licences

Frequently asked

The 2017 paper from eight Google researchers that introduced the transformer: a model that reads a whole sentence at once and learns which words to pay attention to, instead of plodding through it one word at a time. Every large language model since, including the GPT family, is built on it. Free to read on arXiv at arxiv.org/abs/1706.03762.