Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs.
The complete show notes for this episode can be found at twimlai.com/go/663.
Constraint Active Search for Human-in-the-Loop Optimization with Gustavo Malkomes - #505
Fairness and Robustness in Federated Learning with Virginia Smith -#504
Scaling AI at H&M Group with Errol Koolmeister - #503
Evolving AI Systems Gracefully with Stefano Soatto - #502
ML Innovation in Healthcare with Suchi Saria - #501
Cross-Device AI Acceleration, Compilation & Execution with Jeff Gehlhaar - #500
The Future of Human-Machine Interaction with Dan Bohus and Siddhartha Sen - #499
Vector Quantization for NN Compression with Julieta Martinez - #498
Deep Unsupervised Learning for Climate Informatics with Claire Monteleoni - #497
Skip-Convolutions for Efficient Video Processing with Amir Habibian - #496
Advancing NLP with Project Debater w/ Noam Slonim - #495
Bringing AI Up to Speed with Autonomous Racing w/ Madhur Behl - #494
AI and Society: Past, Present and Future with Eric Horvitz - #493
Agile Applied AI Research with Parvez Ahammad - #492
Haptic Intelligence with Katherine J. Kuchenbecker - #491
Data Science on AWS with Chris Fregly and Antje Barth - #490
Accelerating Distributed AI Applications at Qualcomm with Ziad Asghar - #489
Buy AND Build for Production Machine Learning with Nir Bar-Lev - #488
Applied AI Research at AWS with Alex Smola - #487
Causal Models in Practice at Lyft with Sean Taylor - #486
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week