On this episode of Bit of A Tangent, we discuss the emerging field of self-supervised machine learning. This is an immensely exciting area of active research in machine learning and AI - one which most people haven’t even heard about yet! We build up to the intuition for the topic by covering supervised and unsupervised learning; autoencoders and dimensionality reduction, and exploring how these techniques could be applied to Gianluca’s Quantified Self n=1 sleep quality dataset. We culminate in a detailed discussion of the state-of-the-art Contrastive Predictive Coding model, and how it allows us to learn about the structure of the world, without tonnes of labelled training data!
--------
Shownotes:
--------
Jared on Twitter: www.twitter.com/jnearestn
Gianluca on Twitter: www.twitter.com/QVagabond
Bit of a Tangent on Twitter (www.twitter.com/podtangent) and Instagram (instagram.com/podtangent/)
Summer school on Computational Neuroscience: http://imbizo.africa/
Control problem in AI: https://intelligence.org/stanford-talk/
Coordination problem: https://conceptually.org/concepts/coordination-problems
Deep learning overview: https://lilianweng.github.io/lil-log/2017/06/21/an-overview-of-deep-learning.html
t-SNE explained: https://mlexplained.com/2018/09/14/paper-dissected-visualizing-data-using-t-sne-explained/
Variational autoencoders explained: https://anotherdatum.com/vae.html
Self-supervised learning by fast.ai: https://www.fast.ai/2020/01/13/self_supervised/
CPC model papers on Arxiv: https://arxiv.org/pdf/1807.03748.pdf https://arxiv.org/pdf/1905.09272.pdf
Blog posts explaining CPC: https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html
https://yann-leguilly.gitlab.io/post/2019-09-29-representation-learning-with-contrastive-predictive-coding/
https://mf1024.github.io/2019/05/27/contrastive-predictive-coding/
031 | Kat Woods — Effective Altruism, Infohazards, and Brain Hacks
030 | Predictive Processing 3: Neurobiology, Prediction, and Computational Psychiatry
029 | Predictive Processing 2: Where The Action Is
028 | Predictive Processing 1: The World You Thought You Knew
027 | 7 Practices to Survive the 2020 Apocalypse
026 | Drink and be Rational
024 | Replacing Guilt
023 | Are Brains Just Neural Nets?
022 | 6 Books You Need to Read in 2020… and 1 You Probably Shouldn’t
021 | The Top 19 Ideas from 2019
020 | Mental Models 3: Fantastic Biases and Where To Find Them
019 | Mental Models 2: How To Think Like A Bayesian, Avoid Black Swans, & Know The Value of Your Decisions
018 | Mental Models 1: How To Have Better Ideas and Improve Your Thinking
017 | Bronwyn Williams: Universal Basic Income & Individual Sovereignty
016 | Free Will, Compassion, and Reinforcement Learning
015 | You have no Choice in the Matter
014 | The Art of Looking - Part 1 of ??
013 | How To Learn Anything!
012 | How Deep Learning Does Magic
Create your
podcast in
minutes
It is Free
DNA Today: A Genetics Podcast
The Psychic Elephant Radio Podcast
Strange by Nature Podcast
Sasquatch Chronicles
Hidden Brain