How “it” finds “animal.”

Video transcript

Here is our sentence: the animal didn't cross the street because it was tired. Who is it?

We know it means the animal. But how does the model know, if it only sees numbers?

The word it asks every other word, how much do you matter to me? Each match gets a score. Animal scores nine, street three, was one.

And softmax turns those into weights, so animal gets almost everything. If we mix values with these weights, what will the new it look like?

Right. Each word carries a value. We multiply each value by its weight: zero point nine nine seven times animal, a little street, a little was.

Add them up, and it becomes a new vector that is almost entirely animal. That is how it finds animal.