Every word becomes a point in space.

Video transcript

So each token becomes numbers. But what does a point in space have to do with a word?

You know a vector is a list of numbers, like two, seven. That is a point on a graph. Let's say the word cat is that point.

Now place dog and car too. Where would you put them?

Dog should be near cat, since both are pets. Car can go far away.

Exactly. Similar meaning means a small distance between points. Different meaning means a big distance.

But a graph with two numbers seems tiny for all of language.

Right. Real models use hundreds of numbers per word, so many directions can capture meaning. The idea is the same: a word is a point, and neighbors are similar words.