Tai-Danae Bradley | Category Theory and Language Models

Tai-Danae Bradley is a mathematician who received her Ph.D. in mathematics from the CUNY Graduate Center. She was formerly at Alphabet and is now at Sandbox AQ, a startup focused on combining machine learning and quantum physics. Tai-Danae is a visiting research professor of mathematics at The Master’s University and the executive director of the Math3ma Institute, where she hosts her popular blog on category theory. She is also a co-author of the textbook Topology: A Categorical Approach that presents basic topology from the modern perspective of category theory.

In this episode, we provide a compressed crash course in category theory. We provide definitions and plenty of basic examples for all the basic notions: objects, morphisms, categories, functors, natural transformations. We also discuss the first basic result in category theory which is the Yoneda Lemma. We conclude with a discussion of how Tai-Danae has used category-theoretic methods in her work on language modeling, in particular, in how the passing from syntax to semantics can be realized through category-theoretic notions.

Patreon: https://www.patreon.com/timothynguyen

Originally published on July 20, 2022 on YouTube: https://youtu.be/Gz8W1r90olc

Timestamps:

• 00:00:00 : Introduction
• 00:03:07 : How did you get into category theory?
• 00:06:29 : Outline of podcast
• 00:09:21 : Motivating category theory
• 00:11:35 : Analogy: Object Oriented Programming
• 00:12:32 : Definition of category
• 00:18:50 : Example: Category of sets
• 00:20:17 : Example: Matrix category
• 00:25:45 : Example: Preordered set (poset) is a category
• 00:33:43 : Example: Category of finite-dimensional vector spaces
• 00:37:46 : Forgetful functor
• 00:39:15 : Fruity example of forgetful functor: Forget race, gender, we're all part of humanity!
• 00:40:06 : Definition of functor
• 00:42:01 : Example: API change between programming languages is a functor
• 00:44:23 : Example: Groups, group homomorphisms are categories and functors
• 00:47:33 : Resume definition of functor
• 00:49:14 : Example: Functor between poset categories = order-preserving function
• 00:52:28 : Hom Functors. Things are getting meta (no not the tech company)
• 00:57:27 : Category theory is beautiful because of its rigidity
• 01:00:54 : Contravariant functor
• 01:03:23 : Definition: Presheaf
• 01:04:04 : Why are things meta? Arrows, arrows between arrows, categories of categories, ad infinitum.
• 01:07:38 : Probing a space with maps (prelude to Yoneda Lemma)
• 01:12:10 : Algebraic topology motivated category theory
• 01:15:44 : Definition: Natural transformation
• 01:19:21 : Example: Indexing category
• 01:21:54 : Example: Change of currency as natural transformation
• 01:25:35 : Isomorphism and natural isomorphism
• 01:27:34 : Notion of isomorphism in different categories
• 01:30:00 : Yoneda Lemma
• 01:33:46 : Example for Yoneda Lemma: Identity functor and evaluation natural transformation
• 01:42:33 : Analogy between Yoneda Lemma and linear algebra
• 01:46:06 : Corollary of Yoneda Lemma: Isomorphism of objects = Isomorphism of hom functors
• 01:50:40 : Yoneda embedding is fully faithful. Reasoning about this.
• 01:55:15 : Language Category
• 02:03:10 : Tai-Danae's paper: "An enriched category theory of language: from syntax to semantics"

Further Reading:

• Tai-Danae's Blog: https://www.math3ma.com/categories
• Tai-Danae Bradley. "What is applied category theory?" https://arxiv.org/pdf/1809.05923.pdf
• Tai-Danae Bradley, John Terilla, Yiannis Vlassopoulos. "An enriched category theory of language: from syntax to semantics." https://arxiv.org/pdf/2106.07890.pdf