Ep. 1 - The History of Machine Learning with Tom Mitchell | Stanford Digital Economy Lab

The podcast explores the history of machine learning, tracing its evolution from philosophical roots to modern AI systems. It highlights key milestones, including Art Samuel's checkers program, Frank Rosenblatt's perceptrons, and the shift towards symbolic descriptions with programs like Meta-dendral. The integration of statistical methods and the rise of neural networks are discussed, featuring insights from Jeff Hinton on backpropagation and Yann LeCun on self-supervised learning. Dean Pomelo's neural network for self-driving cars exemplifies the innovative spirit of the field. The conversation also covers the impact of PAC learning and the development of reinforcement learning, emphasizing the blend of technical and social forces driving progress.

Outlines

Part 1: Origins and Philosophical Foundations

Part 2: Early Milestones and Initial Paradigms

Part 3: Theoretical Frameworks and the Neural Rebirth

Part 4: Reinforcement Learning and Statistical Integration

Part 5: The Era of Big Data and Deep Learning

Part 6: Reflections and Future Guidance

Sign in to continue reading, translating and more.

Open full episode in Podwise

Ep. 1 - The History of Machine Learning with Tom Mitchell

Stanford Digital Economy Lab

Part 1: Origins and Philosophical Foundations

Machine Learning: A 75-Year Journey from Hand-Written Programs to Trained Intelligence

Philosophical Roots of Machine Learning: Induction, Ockham's Razor, and Hume's Skepticism

Part 2: Early Milestones and Initial Paradigms

Early Machine Learning: Turing's Vision and Samuel's Checkers Program

Neural Networks: Rosenblatt's Perceptron and Minsky & Papert's Critique

Symbolic Learning Emerges: Winston's Arch Program and Buchanan's Meta-Dendral

Version Spaces, Active Learning, and the First Machine Learning Workshop

Part 3: Theoretical Frameworks and the Neural Rebirth

Decision Trees and PAC Learning: Quinlan's ID3 and Valiant's Theoretical Framework

PAC Learning's Impact, SOAR Architecture, and the Rebirth of Neural Networks

Backpropagation, Hinton's Family Trees, and LeCun's Convolutional Nets

Part 4: Reinforcement Learning and Statistical Integration

Self-Driving Cars and Reinforcement Learning: Pomelo's Neural Net and Sutton & Bartow's Framework

Reinforcement Learning's Power and the Integration of Statistics in the 1990s

Graphical Models and the Richly Structured Problems

Statistics and Machine Learning: Two Sides of the Same Coin

Part 5: The Era of Big Data and Deep Learning

The Unreasonable Effectiveness of Data and the 2012 ImageNet Breakthrough

Deep Learning's Rise: From Vision to Language Models with Transformers

Part 6: Reflections and Future Guidance

Scientific Progress in Machine Learning: Themes and Observations

Lessons for Researchers: Question Authority, Communicate, and Understand the Fundamentals

Ep. 1 - The History of Machine Learning with Tom Mitchell

Stanford Digital Economy Lab

Part 1: Origins and Philosophical Foundations

00:00Machine Learning: A 75-Year Journey from Hand-Written Programs to Trained Intelligence

Machine Learning: A 75-Year Journey from Hand-Written Programs to Trained Intelligence

02:06Philosophical Roots of Machine Learning: Induction, Ockham's Razor, and Hume's Skepticism

Philosophical Roots of Machine Learning: Induction, Ockham's Razor, and Hume's Skepticism

Part 2: Early Milestones and Initial Paradigms

04:45Early Machine Learning: Turing's Vision and Samuel's Checkers Program

Early Machine Learning: Turing's Vision and Samuel's Checkers Program

07:54Neural Networks: Rosenblatt's Perceptron and Minsky & Papert's Critique

Neural Networks: Rosenblatt's Perceptron and Minsky & Papert's Critique

10:40Symbolic Learning Emerges: Winston's Arch Program and Buchanan's Meta-Dendral

Symbolic Learning Emerges: Winston's Arch Program and Buchanan's Meta-Dendral

14:30Version Spaces, Active Learning, and the First Machine Learning Workshop

Version Spaces, Active Learning, and the First Machine Learning Workshop

Part 3: Theoretical Frameworks and the Neural Rebirth

19:14Decision Trees and PAC Learning: Quinlan's ID3 and Valiant's Theoretical Framework

Decision Trees and PAC Learning: Quinlan's ID3 and Valiant's Theoretical Framework

23:43PAC Learning's Impact, SOAR Architecture, and the Rebirth of Neural Networks

PAC Learning's Impact, SOAR Architecture, and the Rebirth of Neural Networks

29:07Backpropagation, Hinton's Family Trees, and LeCun's Convolutional Nets

Backpropagation, Hinton's Family Trees, and LeCun's Convolutional Nets

Part 4: Reinforcement Learning and Statistical Integration

34:44Self-Driving Cars and Reinforcement Learning: Pomelo's Neural Net and Sutton & Bartow's Framework

Self-Driving Cars and Reinforcement Learning: Pomelo's Neural Net and Sutton & Bartow's Framework

38:09Reinforcement Learning's Power and the Integration of Statistics in the 1990s

Reinforcement Learning's Power and the Integration of Statistics in the 1990s

41:10Graphical Models and the Richly Structured Problems

Graphical Models and the Richly Structured Problems

44:56Statistics and Machine Learning: Two Sides of the Same Coin

Statistics and Machine Learning: Two Sides of the Same Coin

Part 5: The Era of Big Data and Deep Learning

49:04The Unreasonable Effectiveness of Data and the 2012 ImageNet Breakthrough

The Unreasonable Effectiveness of Data and the 2012 ImageNet Breakthrough

53:57Deep Learning's Rise: From Vision to Language Models with Transformers

Deep Learning's Rise: From Vision to Language Models with Transformers

Part 6: Reflections and Future Guidance

1:01:23Scientific Progress in Machine Learning: Themes and Observations

Scientific Progress in Machine Learning: Themes and Observations

1:04:20Lessons for Researchers: Question Authority, Communicate, and Understand the Fundamentals

Lessons for Researchers: Question Authority, Communicate, and Understand the Fundamentals