Discover how the 'chain of thought' approach makes transformers smarter and how FastGen cuts GPU memory costs without compromising LLM quality. Also, learn about Lory, a fully-differentiable MoE model for language model pre-training, and the release of the largest supervised fine-tuning open-sourced dataset by Alignment Lab...
Discover how the 'chain of thought' approach makes transformers smarter and how FastGen cuts GPU memory costs without compromising LLM quality. Also, learn about Lory, a fully-differentiable MoE model for language model pre-training, and the release of the largest supervised fine-tuning open-sourced dataset by Alignment Lab AI.
Sources:
https://www.marktechpost.com/2024/05/12/how-chain-of-thought-makes-transformers-smarter/
https://www.marktechpost.com/2024/05/12/fastgen-cutting-gpu-memory-costs-without-compromising-on-llm-quality/
https://www.marktechpost.com/2024/05/12/researchers-from-princeton-and-meta-ai-introduce-lory-a-fully-differentiable-moe-model-designed-for-autoregressive-language-model-pre-training/
https://www.marktechpost.com/2024/05/12/alignment-lab-ai-releases-buzz-dataset-the-largest-supervised-fine-tuning-open-sourced-dataset/
Outline:
(00:00:00) Introduction
(00:00:45) How ‘Chain of Thought’ Makes Transformers Smarter
(00:03:23) FastGen: Cutting GPU Memory Costs Without Compromising on LLM Quality
(00:06:51) Researchers from Princeton and Meta AI Introduce ‘Lory’: A Fully-Differentiable MoE Model Designed for Autoregressive Language Model Pre-Training
(00:09:27) Alignment Lab AI Releases ‘Buzz Dataset’: The Largest Supervised Fine-Tuning Open-Sourced Dataset
View more
