Today we’re joined by Sara Hooker, director at Cohere and head of Cohere For AI, Cohere’s research lab. In our conversation with Sara, we explore some of the challenges with multilingual models like poor data quality and tokenization, and how they rely on data augmentation and preference training to address these bottlenecks. We also discuss the disadvantages and the motivating factors behind the Mixture of Experts technique, and the importance of common language between ML researchers and hardware architects to address the pain points in frameworks and create a better cohesion between the distinct communities. Sara also highlights the impact and the emotional connection that language models have created in society, the benefits and the current safety concerns of universal models, and the significance of having grounded conversations to characterize and mitigate the risk and development of AI models. Along the way, we also dive deep into Cohere and Cohere for AI, along with their Aya project, an open science project that aims to build a state-of-the-art multilingual generative language model as well as some of their recent research papers.
The complete show notes for this episode can be found at twimlai.com/go/651.
Synthetic Data Generation for Robotics with Bill Vass - #588
Multi-Device, Multi-Use-Case Optimization with Jeff Gehlhaar - #587
Causal Conceptions of Fairness and their Consequences with Sharad Goel - #586
Brain-Inspired Hardware and Algorithm Co-Design with Melika Payvand - #585
Equivariant Priors for Compressed Sensing with Arash Behboodi - #584
Managing Data Labeling Ops for Success with Audrey Smith - #583
Engineering an ML-Powered Developer-First Search Engine with Richard Socher - #582
On The Path Towards Robot Vision with Aljosa Osep - #581
More Language, Less Labeling with Kate Saenko - #580
Optical Flow Estimation, Panoptic Segmentation, and Vision Transformers with Fatih Porikli - #579
Data Governance for Data Science with Adam Wood - #578
Feature Platforms for Data-Centric AI with Mike Del Balso - #577
The Fallacy of "Ground Truth" with Shayan Mohanty - #576
Principle-centric AI with Adrien Gaidon - #575
Data Debt in Machine Learning with D. Sculley - #574
AI for Enterprise Decisioning at Scale with Rob Walker - #573
Data Rights, Quantification and Governance for Ethical AI with Margaret Mitchell - #572
Studying Machine Intelligence with Been Kim - #571
Advances in Neural Compression with Auke Wiggers - #570
Mixture-of-Experts and Trends in Large-Scale Language Modeling with Irwan Bello - #569
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week