Nils Reimers on Cohere Embedding Models

Weaviate podcast #33.

Thank you so much for watching the 33rd Weaviate Podcast! This episode features one of the heroes of Deep Learning for Search, Nils Reimers! Nils' work on SentenceBERT is one of the foundational works for applying Deep Representation Learning to text search. This is the idea that personally inspired me to work in this field. Having seen the successes of Contrastive Representation Learning for Computer Vision, I was mind-blown by the possibility of this for NLP and text search. In addition to the scientific foundation, the software development of the Sentence Transformers library and BEIR benchmarks has been enormously impactful! It was an honor getting to ask Nils the questions I have about these things, from the role of Data Quality to Intent, Sparse Vectors, Long Document Encoding, Distribution Shift, and many more. I really hope you enjoy the podcast! We are so excited about the Cohere Multilingual embedding model and can't wait to see what else comes out of Cohere and their amazing team!

Cohere Multilingual ML Models with Weaviate: https://weaviate.io/blog/2022/12/Cohe...

Nils Reimers: https://scholar.google.com/citations?...

Mentioned in the podcast,

Cross-Encoders: https://weaviate.io/blog/2022/08/Usin...

How to choose a Sentence Transformer from HuggingFace: https://weaviate.io/blog/2022/10/How-...

Chapters
0:00 Cohere X Weaviate
0:22 Welcome Nils Reimers!
1:18 Origin Story
3:15 Learning Text Embeddings
6:54 Positive and Negative Sampling in Contrastive Learning
13:32 1 Billion Pairs for Text Embedding Optimization
15:44 Impact of Data Quality
18:40 New Cohere Multilingual Model!
24:50 Challenge of Debugging Multilingual Models
28:30 Intent in Search
30:40 Thoughts on ColBERT
33:50 Sparse Vectors in Search
36:17 Long Documents and Multi-Discourse
43:40 Entity Parsing in Query Understanding
46:08 Unknown Words and Distribution Shift
50:07 Re-Vectorizing with Fine-Tuning
53:07 More on Search Interfaces and Intent in Search
55:15 Thank you Nils!