This 85th episode of Learning Machines 101 discusses formal convergence guarantees for a broad class of machine learning algorithms designed to minimize smooth non-convex objective functions using batch learning methods. In particular, a broad class of unsupervised, supervised, and reinforcement machine learning algorithms which iteratively update their parameter vector by adding a perturbation based upon all of the training data. This process is repeated, making a perturbation of the parameter vector based upon all of the training data until a parameter vector is generated which exhibits improved predictive performance. The magnitude of the perturbation at each learning iteration is called the “stepsize” or “learning rate” and the identity of the perturbation vector is called the “search direction”. Simple mathematical formulas are presented based upon research from the late 1960s by Philip Wolfe and G. Zoutendijk that ensure convergence of the generated sequence of parameter vectors. These formulas may be used as the basis for the design of artificially intelligent smart automatic learning rate selection algorithms. The material in this podcast is designed to provide an overview of Chapter 7 of my new book “Statistical Machine Learning” and is based upon material originally presented in Episode 68 of Learning Machines 101! Check out: www.learningmachines101.com for the show notes!!!
LM101-066: How to Solve Constraint Satisfaction Problems using MCMC Methods (Rerun)
LM101-065: How to Design Gradient Descent Learning Machines (Rerun)
LM101-064: Stochastic Model Search and Selection with Genetic Algorithms (Rerun)
LM101-063: How to Transform a Supervised Learning Machine into a Policy Gradient Reinforcement Learning Machine
LM101-062: How to Transform a Supervised Learning Machine into a Value Function Reinforcement Learning Machine
LM101-061: What happened at the Reinforcement Learning Tutorial? (RERUN)
LM101-060: How to Monitor Machine Learning Algorithms using Anomaly Detection Machine Learning Algorithms
LM101-059: How to Properly Introduce a Neural Network
LM101-058: How to Identify Hallucinating Learning Machines using Specification Analysis
LM101-057: How to Catch Spammers using Spectral Clustering
LM101-056: How to Build Generative Latent Probabilistic Topic Models for Search Engine and Recommender System Applications
LM101-055: How to Learn Statistical Regularities using MAP and Maximum Likelihood Estimation (Rerun)
LM101-054: How to Build Search Engine and Recommender Systems using Latent Semantic Analysis (RERUN)
LM101-053: How to Enhance Learning Machines with Swarm Intelligence (Particle Swarm Optimization)
LM101-052: How to Use the Kernel Trick to Make Hidden Units Disappear
LM101-051: How to Use Radial Basis Function Perceptron Software for Supervised Learning[Rerun]
LM101-050: How to Use Linear Machine Learning Software to Make Predictions (Linear Regression Software)[RERUN]
LM101-049: How to Experiment with Lunar Lander Software
LM101-048: How to Build a Lunar Lander Autopilot Learning Machine (Rerun)
LM101-047: How Build a Support Vector Machine to Classify Patterns (Rerun)
Create your
podcast in
minutes
It is Free
Insight Story: Tech Trends Unpacked
Zero-Shot
Fast Forward by Tomorrow Unlocked: Tech past, tech future
The Unbelivable Truth - Series 1 - 26 including specials and pilot
A Prairie Home Companion: News from Lake Wobegon