Math vs AI: Who Decides What's True? [Dr. Paul Lessard]

The podcast features a discussion on the philosophical underpinnings and practical applications of machine learning, touching on concepts like Platonism, constructivism, and the role of benchmarks in evaluating model performance. The speakers delve into the limitations of current deep learning models, likening them to "sandcastles" due to their lack of inherent structure, and explore the potential of category theory to provide a more robust framework for designing and understanding neural networks. They also discuss the importance of "anything goes" approach in machine learning research, the need for higher-order aesthetic sensibilities in evaluating models, and the challenge of creating adequate explanations for why certain models work. The conversation extends to the role of stochastic processes in machine learning, viewing neural networks as algebras for constructing machines that mimic physical systems, and the importance of curriculum design in education to prepare students for tackling unsolvable problems.

Outlines

Part 1: Truth, Benchmarks, and Explanations

Part 2: Structure, Constraints, and Representations

Part 3: Learning, Equivariance, and Stochastic Processes

Sign in to continue reading, translating and more.

Continue

Machine Learning Street Talk

Part 1: Truth, Benchmarks, and Explanations

The Nature of Truth in Mathematics and Machine Learning

Category Theory, Machine Learning, and the Role of Benchmarks

The "Anything Goes" Approach in Machine Learning and the Search for Adequate Explanations

Part 2: Structure, Constraints, and Representations

Chain of Thought, Sparsity, and the Pursuit of Certainty

Formal Constraints, Category Theory, and the Transformer Architecture

Unified Representations, Learning, and the Challenge of Unsolvable Problems

Part 3: Learning, Equivariance, and Stochastic Processes

Pavlovian Anxiety, Category Theory, and Equivariance in Neural Networks

Enforcing Equivariances, Categorical Deep Learning, and the Stochastic Nature of Machine Learning

Math vs AI: Who Decides What's True? [Dr. Paul Lessard]

Machine Learning Street Talk

Part 1: Truth, Benchmarks, and Explanations

00:00The Nature of Truth in Mathematics and Machine Learning

The Nature of Truth in Mathematics and Machine Learning

04:52Category Theory, Machine Learning, and the Role of Benchmarks

Category Theory, Machine Learning, and the Role of Benchmarks

08:35The "Anything Goes" Approach in Machine Learning and the Search for Adequate Explanations

The "Anything Goes" Approach in Machine Learning and the Search for Adequate Explanations

Part 2: Structure, Constraints, and Representations

13:53Chain of Thought, Sparsity, and the Pursuit of Certainty

Chain of Thought, Sparsity, and the Pursuit of Certainty

17:54Formal Constraints, Category Theory, and the Transformer Architecture

Formal Constraints, Category Theory, and the Transformer Architecture

25:55Unified Representations, Learning, and the Challenge of Unsolvable Problems

Unified Representations, Learning, and the Challenge of Unsolvable Problems

Part 3: Learning, Equivariance, and Stochastic Processes

32:44Pavlovian Anxiety, Category Theory, and Equivariance in Neural Networks

Pavlovian Anxiety, Category Theory, and Equivariance in Neural Networks

39:08Enforcing Equivariances, Categorical Deep Learning, and the Stochastic Nature of Machine Learning

Enforcing Equivariances, Categorical Deep Learning, and the Stochastic Nature of Machine Learning