Today we’re joined by Sanmi Koyejo, assistant professor at Stanford University, to continue our NeurIPS 2024 series. In our conversation, Sanmi discusses his two recent award-winning papers. First, we dive into his paper, “Are Emergent Abilities of Large Language Models a Mirage?” We discuss the different ways LLMs are evaluated and the excitement surrounding their“emergent abilities” such as the ability to perform arithmetic Sanmi describes how evaluating model performance using nonlinear metrics can lead to the illusion that the model is rapidly gaining new capabilities, whereas linear metrics show smooth improvement as expected, casting doubt on the significance of emergence. We continue on to his next paper, “DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models,” discussing the methodology it describes for evaluating concerns such as the toxicity, privacy, fairness, and robustness of LLMs.
The complete show notes for this episode can be found at twimlai.com/go/671.
Scaling Enterprise ML in 2020: Still Hard! with Sushil Thomas - #429
Enabling Clinical Automation: From Research to Deployment with Devin Singh - #428
Pixels to Concepts with Backpropagation w/ Roland Memisevic - #427
Fighting Global Health Disparities with AI w/ Jon Wang - #426
Accessibility and Computer Vision - #425
NLP for Equity Investing with Frank Zhao - #424
The Future of Education and AI with Salman Khan - #423
Why AI Innovation and Social Impact Go Hand in Hand with Milind Tambe - #422
What's Next for Fast.ai? w/ Jeremy Howard - #421
Feature Stores for MLOps with Mike del Balso - #420
Exploring Causality and Community with Suzana Ilić - #419
Decolonizing AI with Shakir Mohamed - #418
Spatial Analysis for Real-Time Video Processing with Adina Trufinescu
How Deep Learning has Revolutionized OCR with Cha Zhang - #416
Machine Learning for Food Delivery at Global Scale - #415
Open Source at Qualcomm AI Research with Jeff Gehlhaar and Zahra Koochak - #414
Visualizing Climate Impact with GANs w/ Sasha Luccioni - #413
ML-Powered Language Learning at Duolingo with Burr Settles - #412
Bridging The Gap Between Machine Learning and the Life Sciences with Artur Yakimovich - #411
Understanding Cultural Style Trends with Computer Vision w/ Kavita Bala - #410
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week