Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs.
The complete show notes for this episode can be found at twimlai.com/go/663.
Applied AI Research at AWS with Alex Smola - #487
Causal Models in Practice at Lyft with Sean Taylor - #486
Using AI to Map the Human Immune System w/ Jabran Zahid - #485
Learning Long-Time Dependencies with RNNs w/ Konstantin Rusch - #484
What the Human Brain Can Tell Us About NLP Models with Allyson Ettinger - #483
Probabilistic Numeric CNNs with Roberto Bondesan - #482
Building a Unified NLP Framework at LinkedIn with Huiji Gao - #481
Dask + Data Science Careers with Jacqueline Nolis - #480
Machine Learning for Equitable Healthcare Outcomes with Irene Chen - #479
AI Storytelling Systems with Mark Riedl - #478
Creating Robust Language Representations with Jamie Macbeth - #477
Reinforcement Learning for Industrial AI with Pieter Abbeel - #476
AutoML for Natural Language Processing with Abhishek Thakur - #475
Inclusive Design for Seeing AI with Saqib Shaikh - #474
Theory of Computation with Jelani Nelson - #473
Human-Centered ML for High-Risk Behaviors with Stevie Chancellor - #472
Operationalizing AI at Dataiku with Conor Jensen - #471
ML Lifecycle Management at Algorithmia with Diego Oppenheimer - #470
End to End ML at Cloudera with Santiago Giraldo - #469 [TWIMLcon Sponsor Series]
ML Platforms for Global Scale at Prosus with Paul van der Boor - #468 [TWIMLcon Sponsor Series]
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week