Episode 11.83: More attention than you need; Query, Key and Value embeddings almost explained.
Unmaking Sense

Episode 11.83: More attention than you need; Query, Key and Value embeddings almost explained.

2024-04-07
Going beyond mere token input embeddings to the attention, self-attention and multi-head attention process that lies at the heart of transformer architecture and generative prompt/completion models. The central section contains minor additions to the first and third.
Comments (3)

More Episodes

All Episodes>>

Get this podcast on your phone, Free

Create Your Podcast In Minutes

  • Full-featured podcast site
  • Unlimited storage and bandwidth
  • Comprehensive podcast stats
  • Distribute to Apple Podcasts, Spotify, and more
  • Make money with your podcast
Get Started
It is Free