From numbers back to a word.

Video transcript

The decoder finishes with a vector of numbers. But we need a real French word. How do we get back?

A final layer gives every word in the vocabulary a score. Higher means a better fit for the next spot.

But scores can be negative or any size. They aren't percentages. What do we do about that?

Softmax. It makes every score positive and makes them all add up to one, so they become probabilities.

So the biggest score still gets the biggest share. Then we just pick the best one, chat?

Yes. Pick the highest probability, and numbers become a word. Then that word feeds back in for the next step.