Today we’re joined by Sara Hooker, director at Cohere and head of Cohere For AI, Cohere’s research lab. In our conversation with Sara, we explore some of the challenges with multilingual models like poor data quality and tokenization, and how they rely on data augmentation and preference training to address these bottlenecks. We also discuss the disadvantages and the motivating factors behind the Mixture of Experts technique, and the importance of common language between ML researchers and hardware architects to address the pain points in frameworks and create a better cohesion between the distinct communities. Sara also highlights the impact and the emotional connection that language models have created in society, the benefits and the current safety concerns of universal models, and the significance of having grounded conversations to characterize and mitigate the risk and development of AI models. Along the way, we also dive deep into Cohere and Cohere for AI, along with their Aya project, an open science project that aims to build a state-of-the-art multilingual generative language model as well as some of their recent research papers.
The complete show notes for this episode can be found at twimlai.com/go/651.
Kids Run the Darndest Experiments: Causal Learning in Children with Alison Gopnik - #548
Hypergraphs, Simplicial Complexes and Graph Representations of Complex Systems with Tina Eliassi-Rad - #547
Deep Learning, Transformers, and the Consequences of Scale with Oriol Vinyals - #546
Optimization, Machine Learning and Intelligent Experimentation with Michael McCourt - #545
Jupyter and the Evolution of ML Tooling with Brian Granger - #544
Creating a Data-Driven Culture at ADP with Jack Berkowitz - #543
re:Invent Roundup 2021 with Bratin Saha - #542
Multi-modal Deep Learning for Complex Document Understanding with Doug Burdick - #541
Predictive Maintenance Using Deep Learning and Reliability Engineering with Shayan Mortazavi - #540
Building a Deep Tech Startup in NLP with Nasrin Mostafazadeh - #539
Models for Human-Robot Collaboration with Julie Shah - #538
Four Key Tools for Robust Enterprise NLP with Yunyao Li - #537
Machine Learning at GSK with Kim Branson - #536
The Benefit of Bottlenecks in Evolving Artificial Intelligence with David Ha - #535
Facebook Abandons Facial Recognition. Should Everyone Else Follow Suit? With Luke Stark - #534
Building Blocks of Machine Learning at LEGO with Francesc Joan Riera - #533
Exploring the FastAI Tooling Ecosystem with Hamel Husain - #532
Multi-task Learning for Melanoma Detection with Julianna Ianni - #531
House Hunters: Machine Learning at Redfin with Akshat Kaul - #530
Attacking Malware with Adversarial Machine Learning, w/ Edward Raff - #529
Create your
podcast in
minutes
It is Free
20/20
The Dropout
Ten Percent Happier with Dan Harris
World News Tonight with David Muir
NEJM This Week