Episode 8.52: Attention is all you need: how tokens find matches that will come next.

The “Attention is All You Need” paper lies behind Andrew Karpathy’s excellent YouTube video “Let’s build GPT: from scratch, in code, spelled out”. We discuss some implications.