How does it learn? Guess, check, fix.

Video transcript

The model reads a sentence and guesses the next French word. Say it guesses chien, but the truth is chat.

We measure how wrong it was with a number called the loss. Big when the right word gets a tiny probability, small when it gets a big one.

So learning is just minimizing an error? Like finding the lowest point of a curve?

Exactly. Imagine loss as a curve over the model's settings. The gradient is the slope. It tells us which way is uphill.

Then we step the opposite way, downhill. But how big a step should we take?

Small. Nudge each setting a little against the gradient, then guess again. The loss drops a bit each time.

Do this over millions of sentences, and the model gets good. Guess, check, fix, repeat. How big the step is comes next.