skylabGeneral · LESSON 8 OF 33 · 1 min
What is a GPU? 60 seconds, no jargon.
About this lesson
What is a GPU? 60 seconds, no jargon. — a video lesson in attention is all you need. Learn attention is all you need through short video lessons.
Continue in the reels feed ↓Video transcript
People keep saying the paper needs a GPU. What is that?
A normal computer chip is like one very fast worker. It does one hard job after another.
A GPU is different. It has thousands of tiny calculators. Each one is simple, but there are so many.
So if I give them a huge pile of the same math, they all work at once? Would that be faster for a long list of numbers?
Yes. Same math on many numbers is perfect. Tables of numbers, called matrices, are exactly that, and a sentence turns into matrices.
That is why the paper dropped the one-word-at-a-time line. It lets every word work at the same moment on a GPU.