Today we’re joined by Sara Hooker, director at Cohere and head of Cohere For AI, Cohere’s research lab. In our conversation with Sara, we explore some of the challenges with multilingual models like poor data quality and tokenization, and how they rely on data augmentation and preference training to address these bottlenecks. We also discuss the disadvantages and the motivating factors behind the Mixture of Experts technique, and the importance of common language between ML researchers and hardware architects to address the pain points in frameworks and create a better cohesion between the distinct communities. Sara also highlights the impact and the emotional connection that language models have created in society, the benefits and the current safety concerns of universal models, and the significance of having grounded conversations to characterize and mitigate the risk and development of AI models. Along the way, we also dive deep into Cohere and Cohere for AI, along with their Aya project, an open science project that aims to build a state-of-the-art multilingual generative language model as well as some of their recent research papers.
The complete show notes for this episode can be found at twimlai.com/go/651.
Learning Transformer Programs with Dan Friedman - #667
AI Trends 2024: Machine Learning & Deep Learning with Thomas Dietterich - #666
AI Trends 2024: Computer Vision with Naila Murray - #665
Are Vector DBs the Future Data Platform for AI? with Ed Anuff - #664
Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663
Responsible AI in the Generative Era with Michael Kearns - #662
Edutainment for AI and AWS PartyRock with Mike Miller - #661
Data, Systems and ML for Visual Understanding with Cody Coleman - #660
Patterns and Middleware for LLM Applications with Kyle Roche - #659
AI Access and Inclusivity as a Technical Challenge with Prem Natarajan - #658
Building LLM-Based Applications with Azure OpenAI with Jay Emery - #657
Visual Generative AI Ecosystem Challenges with Richard Zhang - #656
Deploying Edge and Embedded AI Systems with Heather Gorr - #655
AI Sentience, Agency and Catastrophic Risk with Yoshua Bengio - #654
Delivering AI Systems in Highly Regulated Environments with Miriam Friedel - #653
Mental Models for Advanced ChatGPT Prompting with Riley Goodside - #652
Scaling Multi-Modal Generative AI with Luke Zettlemoyer - #650
Pushing Back on AI Hype with Alex Hanna - #649
Personalization for Text-to-Image Generative AI with Nataniel Ruiz - #648
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week