Teaching order with sine waves.

Video transcript

So we need to stamp each word with its position. Why not just add the numbers one, two, three?

Raw numbers grow huge in long texts. Waves stay between minus one and one. Here is a slow sine wave. Each position reads a different height from it.

Now add a faster wave. It wiggles quickly, so it reads very different heights for neighbors.

So position one, two, three, four each land on different spots on both waves. Could two positions ever match on both?

Very unlikely. One wave may repeat a height, but with many speeds together, the combination is unique for each position.

Like clock hands! Seconds, minutes, and hours together tell the exact time.

Exactly. Each word gets a vector of wave values, and we add it to the word's meaning. Now attention can tell order.