Today we’re joined by Sanmi Koyejo, assistant professor at Stanford University, to continue our NeurIPS 2024 series. In our conversation, Sanmi discusses his two recent award-winning papers. First, we dive into his paper, “Are Emergent Abilities of Large Language Models a Mirage?” We discuss the different ways LLMs are evaluated and the excitement surrounding their“emergent abilities” such as the ability to perform arithmetic Sanmi describes how evaluating model performance using nonlinear metrics can lead to the illusion that the model is rapidly gaining new capabilities, whereas linear metrics show smooth improvement as expected, casting doubt on the significance of emergence. We continue on to his next paper, “DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models,” discussing the methodology it describes for evaluating concerns such as the toxicity, privacy, fairness, and robustness of LLMs.
The complete show notes for this episode can be found at twimlai.com/go/671.
Open Source Generative AI at Hugging Face with Jeff Boudier - #624
Generative AI at the Edge with Vinesh Sukumar - #623
Runway Gen-2: Generative AI for Video Creation with Anastasis Germanidis - #622
Watermarking Large Language Models to Fight Plagiarism with Tom Goldstein - 621
Does ChatGPT “Think”? A Cognitive Neuroscience Perspective with Anna Ivanova - #620
Robotic Dexterity and Collaboration with Monroe Kennedy III - #619
Privacy and Security for Stable Diffusion and LLMs with Nicholas Carlini - #618
Understanding AI’s Impact on Social Disparities with Vinodkumar Prabhakaran - #617
AI Trends 2023: Causality and the Impact on Large Language Models with Robert Osazuwa Ness - #616
Data-Centric Zero-Shot Learning for Precision Agriculture with Dimitris Zermas - #615
How LLMs and Generative AI are Revolutionizing AI for Science with Anima Anandkumar - #614
AI Trends 2023: Natural Language Proc - ChatGPT, GPT-4 and Cutting Edge Research with Sameer Singh - #613
AI Trends 2023: Reinforcement Learning - RLHF, Robotic Pre-Training, and Offline RL with Sergey Levine - #612
Supporting Food Security in Africa Using ML with Catherine Nakalembe - #611
Service Cards and ML Governance with Michael Kearns - #610
Reinforcement Learning for Personalization at Spotify with Tony Jebara - #609
Will ChatGPT take my job? - #608
Geospatial Machine Learning at AWS with Kumar Chellapilla - #607
Real-Time ML Workflows at Capital One with Disha Singla - #606
Weakly Supervised Causal Representation Learning with Johann Brehmer - #605
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week