General
attention is all you needComputers can’t read. So how do they translate?
Computers can’t read. So how do they translate?
Computers don’t read words. They read pieces.
Computers don’t read words. They read pieces.
Every word becomes a point in space.
Every word becomes a point in space.
“Bank” means two things. How does a machine pick one?
“Bank” means two things. How does a machine pick one?
The old way: read one word at a time.
The old way: read one word at a time.
Why the old way forgets your first sentence.
Why the old way forgets your first sentence.
Why the old way is painfully slow.
Why the old way is painfully slow.
What is a GPU? 60 seconds, no jargon.
What is a GPU? 60 seconds, no jargon.
What if every word could look at every other word?
What if every word could look at every other word?
Attention is a search engine inside the sentence.
Attention is a search engine inside the sentence.
How a word finds its match.
How a word finds its match.
Turning scores into percentages.
Turning scores into percentages.
How “it” finds “animal.”
How “it” finds “animal.”
Where do Q, K, V come from?
Where do Q, K, V come from?
The tiny √d fix that saves everything.
The tiny √d fix that saves everything.
All of attention in one line of math.
All of attention in one line of math.
One look isn’t enough. Use 8.
One look isn’t enough. Use 8.
The flaw: it can’t tell “dog bites man” from “man bites dog.”
The flaw: it can’t tell “dog bites man” from “man bites dog.”
Teaching order with sine waves.
Teaching order with sine waves.
The trick that lets you stack many layers.
The trick that lets you stack many layers.
Attention mixes. This part thinks.
Attention mixes. This part thinks.
The encoder: how it reads a sentence.
The encoder: how it reads a sentence.
The decoder: writing one word at a time.
The decoder: writing one word at a time.
No peeking: the mask.
No peeking: the mask.
How the writer looks back at the reader.
How the writer looks back at the reader.
From numbers back to a word.
From numbers back to a word.
How does it learn? Guess, check, fix.
How does it learn? Guess, check, fix.
Start slow, speed up, slow down.
Start slow, speed up, slow down.
Why this trains so much faster.
Why this trains so much faster.
The scoreboard: did it actually work?
The scoreboard: did it actually work?
What do the heads actually look at?
What do the heads actually look at?
One paper. Then ChatGPT.
One paper. Then ChatGPT.
Explain the whole paper in 60 seconds.
Explain the whole paper in 60 seconds.
YOU’VE REACHED THE END
Keep your curiosity going.
0 of 33 lessons completed.