How the writer looks back at the reader.

Video transcript

The encoder has already read the English sentence. The decoder is writing French, one word at a time.

So how does the writer use what the reader understood? Does it just copy the whole sentence?

No. It looks back. The decoder makes a query, a question like, which English word do I need now? The encoder holds keys and values for every English word.

So the query is compared with each key, and the scores become percentages. When writing the word after Le, which English word should get the biggest share?

Yes, cat. The big match pulls in cat's value, its meaning, and mostly ignores the other words.

So the writer asks, the reader answers. That is cross-attention, and nothing is masked, because the whole English sentence is known.