Today, we continue our NeurIPS series with Dan Friedman, a PhD student in the Princeton NLP group. In our conversation, we explore his research on mechanistic interpretability for transformer models, specifically his paper, Learning Transformer Programs. The LTP paper proposes modifications to the transformer architecture which allow transformer models to be easily converted into human-readable programs, making them inherently interpretable. In our conversation, we compare the approach proposed by this research with prior approaches to understanding the models and their shortcomings. We also dig into the approach’s function and scale limitations and constraints.
The complete show notes for this episode can be found at twimlai.com/go/667.
Scaling Enterprise ML in 2020: Still Hard! with Sushil Thomas - #429
Enabling Clinical Automation: From Research to Deployment with Devin Singh - #428
Pixels to Concepts with Backpropagation w/ Roland Memisevic - #427
Fighting Global Health Disparities with AI w/ Jon Wang - #426
Accessibility and Computer Vision - #425
NLP for Equity Investing with Frank Zhao - #424
The Future of Education and AI with Salman Khan - #423
Why AI Innovation and Social Impact Go Hand in Hand with Milind Tambe - #422
What's Next for Fast.ai? w/ Jeremy Howard - #421
Feature Stores for MLOps with Mike del Balso - #420
Exploring Causality and Community with Suzana Ilić - #419
Decolonizing AI with Shakir Mohamed - #418
Spatial Analysis for Real-Time Video Processing with Adina Trufinescu
How Deep Learning has Revolutionized OCR with Cha Zhang - #416
Machine Learning for Food Delivery at Global Scale - #415
Open Source at Qualcomm AI Research with Jeff Gehlhaar and Zahra Koochak - #414
Visualizing Climate Impact with GANs w/ Sasha Luccioni - #413
ML-Powered Language Learning at Duolingo with Burr Settles - #412
Bridging The Gap Between Machine Learning and the Life Sciences with Artur Yakimovich - #411
Understanding Cultural Style Trends with Computer Vision w/ Kavita Bala - #410
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week