Today we’re joined by Ben Prystawski, a PhD student in the Department of Psychology at Stanford University working at the intersection of cognitive science and machine learning. Our conversation centers on Ben’s recent paper, “Why think step by step? Reasoning emerges from the locality of experience,” which he recently presented at NeurIPS 2023. In this conversation, we start out exploring basic questions about LLM reasoning, including whether it exists, how we can define it, and how techniques like chain-of-thought reasoning appear to strengthen it. We then dig into the details of Ben’s paper, which aims to understand why thinking step-by-step is effective and demonstrates that local structure is the key property of LLM training data that enables it.
The complete show notes for this episode can be found at twimlai.com/go/673.
Open Source Generative AI at Hugging Face with Jeff Boudier - #624
Generative AI at the Edge with Vinesh Sukumar - #623
Runway Gen-2: Generative AI for Video Creation with Anastasis Germanidis - #622
Watermarking Large Language Models to Fight Plagiarism with Tom Goldstein - 621
Does ChatGPT “Think”? A Cognitive Neuroscience Perspective with Anna Ivanova - #620
Robotic Dexterity and Collaboration with Monroe Kennedy III - #619
Privacy and Security for Stable Diffusion and LLMs with Nicholas Carlini - #618
Understanding AI’s Impact on Social Disparities with Vinodkumar Prabhakaran - #617
AI Trends 2023: Causality and the Impact on Large Language Models with Robert Osazuwa Ness - #616
Data-Centric Zero-Shot Learning for Precision Agriculture with Dimitris Zermas - #615
How LLMs and Generative AI are Revolutionizing AI for Science with Anima Anandkumar - #614
AI Trends 2023: Natural Language Proc - ChatGPT, GPT-4 and Cutting Edge Research with Sameer Singh - #613
AI Trends 2023: Reinforcement Learning - RLHF, Robotic Pre-Training, and Offline RL with Sergey Levine - #612
Supporting Food Security in Africa Using ML with Catherine Nakalembe - #611
Service Cards and ML Governance with Michael Kearns - #610
Reinforcement Learning for Personalization at Spotify with Tony Jebara - #609
Will ChatGPT take my job? - #608
Geospatial Machine Learning at AWS with Kumar Chellapilla - #607
Real-Time ML Workflows at Capital One with Disha Singla - #606
Weakly Supervised Causal Representation Learning with Johann Brehmer - #605
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week