Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs.
The complete show notes for this episode can be found at twimlai.com/go/663.
Off-Line, Off-Policy RL for Real-World Decision Making at Facebook - #448
A Future of Work for the Invisible Workers in A.I. with Saiph Savage - #447
Trends in Graph Machine Learning with Michael Bronstein - #446
Trends in Natural Language Processing with Sameer Singh - #445
Trends in Computer Vision with Pavan Turaga - #444
Trends in Reinforcement Learning with Pablo Samuel Castro - #443
MOReL: Model-Based Offline Reinforcement Learning with Aravind Rajeswaran - #442
Machine Learning as a Software Engineering Enterprise with Charles Isbell - #441
Natural Graph Networks with Taco Cohen - #440
Productionizing Time-Series Workloads at Siemens Energy with Edgar Bahilo Rodriguez - #439
ML Feature Store at Intuit with Srivathsan Canchi - #438
re:Invent Roundup 2020 with Swami Sivasubramanian - #437
Predictive Disease Risk Modeling at 23andMe with Subarna Sinha - #436
Scaling Video AI at RTL with Daan Odijk - #435
Benchmarking ML with MLCommons w/ Peter Mattson - #434
Deep Learning for NLP: From the Trenches with Charlene Chambliss - #433
Feature Stores for Accelerating AI Development - #432
An Exploration of Coded Bias with Shalini Kantayya, Deb Raji and Meredith Broussard - #431
Common Sense as an Algorithmic Framework with Dileep George - #430
Scaling Enterprise ML in 2020: Still Hard! with Sushil Thomas - #429
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week