Attention is a search engine inside the sentence.

Video transcript

Last time, every word could look at every word. But how does a word know which words matter? Think of a search engine.

Each word sends out a question, called the query. Tired asks: who is the one that is tired?

So who answers? Do the other words have something to answer with?

Yes. Every word holds a label, called its key, like a page title. Animal's label says a creature. Which word should the query match best?

Right, animal. When the query matches a key, that word hands back its value, the actual information it carries.

So query asks, key is the label, value is the content that comes back. Just like searching!