Today we’re joined by Sara Hooker, director at Cohere and head of Cohere For AI, Cohere’s research lab. In our conversation with Sara, we explore some of the challenges with multilingual models like poor data quality and tokenization, and how they rely on data augmentation and preference training to address these bottlenecks. We also discuss the disadvantages and the motivating factors behind the Mixture of Experts technique, and the importance of common language between ML researchers and hardware architects to address the pain points in frameworks and create a better cohesion between the distinct communities. Sara also highlights the impact and the emotional connection that language models have created in society, the benefits and the current safety concerns of universal models, and the significance of having grounded conversations to characterize and mitigate the risk and development of AI models. Along the way, we also dive deep into Cohere and Cohere for AI, along with their Aya project, an open science project that aims to build a state-of-the-art multilingual generative language model as well as some of their recent research papers.
The complete show notes for this episode can be found at twimlai.com/go/651.
Geospatial Machine Learning at AWS with Kumar Chellapilla - #607
Real-Time ML Workflows at Capital One with Disha Singla - #606
Weakly Supervised Causal Representation Learning with Johann Brehmer - #605
Stable Diffusion & Generative AI with Emad Mostaque - #604
Exploring Large Language Models with ChatGPT - #603
Accelerating Intelligence with AI-Generating Algorithms with Jeff Clune - #602
Programmatic Labeling and Data Scaling for Autonomous Commercial Aviation with Cedric Cocaud - #601
Engineering Production NLP Systems at T-Mobile with Heather Nolis - #600
Sim2Real and Optimus, the Humanoid Robot with Ken Goldberg - #599
The Evolution of the NLP Landscape with Oren Etzioni - #598
Live from TWIMLcon! The Great MLOps Debate: End-to-End ML Platforms vs Specialized Tools - #597
Live from TWIMLcon! You're not Facebook. Architecting MLOps for B2B Use Cases with Jacopo Tagliabue - #596
Building Foundational ML Platforms with Kubernetes and Kubeflow with Ali Rodell - #595
AI-Powered Peer Programming with Vasi Philomin - #594
The Top 10 Reasons to Register for TWIMLcon: AI Platforms 2022!
Applied AI/ML Research at PayPal with Vidyut Naware - #593
Assessing Data Quality at Shopify with Wendy Foster - #592
Transformers for Tabular Data at Capital One with Bayan Bruss - #591
Understanding Collective Insect Communication with ML, w/ Orit Peleg - #590
Multimodal, Multi-Lingual NLP at Hugging Face with John Bohannon and Douwe Kiela - #589
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week