How a word finds its match.

Video transcript

You know the dot product as a similarity score. Here is the query as an arrow. Tired is asking its question.

And each word's key is also an arrow, right? So we compare the query with every key?

Exactly. Take the dot product of the query with each key. Arrows pointing the same way give a big number. Different directions give a small one.

So animal scores nine, because its key points the same way as the query. Was only scores one.

That's how a word finds its match: the highest dot product wins. Next, we turn these raw scores into percentages.