On this episode of Bit of A Tangent, we discuss the emerging field of self-supervised machine learning. This is an immensely exciting area of active research in machine learning and AI - one which most people haven’t even heard about yet! We build up to the intuition for the topic by covering supervised and unsupervised learning; autoencoders and dimensionality reduction, and exploring how these techniques could be applied to Gianluca’s Quantified Self n=1 sleep quality dataset. We culminate in a detailed discussion of the state-of-the-art Contrastive Predictive Coding model, and how it allows us to learn about the structure of the world, without tonnes of labelled training data!
--------
Shownotes:
--------
Jared on Twitter: www.twitter.com/jnearestn
Gianluca on Twitter: www.twitter.com/QVagabond
Bit of a Tangent on Twitter (www.twitter.com/podtangent) and Instagram (instagram.com/podtangent/)
Summer school on Computational Neuroscience: http://imbizo.africa/
Control problem in AI: https://intelligence.org/stanford-talk/
Coordination problem: https://conceptually.org/concepts/coordination-problems
Deep learning overview: https://lilianweng.github.io/lil-log/2017/06/21/an-overview-of-deep-learning.html
t-SNE explained: https://mlexplained.com/2018/09/14/paper-dissected-visualizing-data-using-t-sne-explained/
Variational autoencoders explained: https://anotherdatum.com/vae.html
Self-supervised learning by fast.ai: https://www.fast.ai/2020/01/13/self_supervised/
CPC model papers on Arxiv: https://arxiv.org/pdf/1807.03748.pdf https://arxiv.org/pdf/1905.09272.pdf
Blog posts explaining CPC: https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html
https://yann-leguilly.gitlab.io/post/2019-09-29-representation-learning-with-contrastive-predictive-coding/
https://mf1024.github.io/2019/05/27/contrastive-predictive-coding/
011 | Slack, attention and focus - or why more is less
010 | Flow states, optimal performance, and PhD hunter-gathering
009 | How to Win Friends and Influence Reality
008 | Design Your Workouts, but Evolve Your Algorithms!
007 | Game Theory, Alignment, and Civilisational Inadequacy
006 | Libra, crypto, and the future of humanity
005 | Habit Frameworks and Sleep
004 | Don't Ask a Genie to Cure Cancer
003 | Synaesthesia and the Butterfly Effect of Remembrance
002 | Self-aware doorways & dog-space
001 | You don’t want to be the people in Friends
Create your
podcast in
minutes
It is Free
DNA Today: A Genetics Podcast
Museum of the Missing
Strange by Nature Podcast
Sasquatch Chronicles
Hidden Brain