Episode 8.90: Teaching GPT new stuff using embedding rather than fine-tuning.

Almost all the GPT models only use training data that goes up to September 2021. As you can scarcely fail to have noticed rather a lot has happened since then. How do you teach GPT new stuff and try to bring it up to date if you want to use it for something that happened after September 2021? The answer is you use a technique called search and ask that uses embeddings to compare the knowledge that you feedit with the questions that you ask it. Details are in the episode.