The ghost in the machine.

The podcast features an interview with Noam Chomsky, focusing on his perspectives on language, cognition, and artificial intelligence. Chomsky argues that large language models have achieved "zero" in terms of genuine understanding, likening them to a physics theory that explains nothing. He critiques the empiricist tradition and emphasizes the importance of innate knowledge and universal grammar, suggesting that language and thought are fundamentally linked. The discussion touches on the limits of human understanding, the nature of semantics, and the role of creativity and free will in language use, with Chomsky advocating for a shift towards deeper explanations rather than superficial approximations in cognitive science. The hosts also discuss the technical challenges of restoring the recording of the interview.

Outlines

Part 1: Framing the Debate: LeCun vs. Chomsky

Part 2: Deep Learning, Symbols, and Logic

Part 3: Philosophical Foundations of AI

Part 4: Limits of Cognition and Science

Part 5: Connectionism vs. Symbolic Representation

Part 6: The Chomsky Interview: LLMs and Language

Part 7: Intelligence, Biology, and Universal Grammar

Part 8: Consciousness and Final Reflections

Sign in to continue reading, translating and more.

Open full episode in Podwise

Machine Learning Street Talk

Part 1: Framing the Debate: LeCun vs. Chomsky

Introduction to Noam Chomsky, Radical Empiricism, and the Antithesis of Jan LeCun

Yann LeCun's Path Towards Autonomous Machine Intelligence: A Critical Analysis

The Limits of Perception-Derived Knowledge and the Need for Abstraction

Part 2: Deep Learning, Symbols, and Logic

Compositionality, Symbolic Logic, and the Limitations of Deep Learning

Jürgen Schmidhuber's Critique of LeCun's Paper and the Debate Over Attribution

Energy-Based Models vs. Probability Theory: A Debate on Uncertainty Quantification

Part 3: Philosophical Foundations of AI

World Models, Symbolic Systems, and the Empiricism vs. Rationalism Debate

Empiricism, Rationalism, and the Role of Innate Knowledge in Cognition

Cognitive Templates, Surprise, and the Limits of Empiricism

Part 4: Limits of Cognition and Science

Gödel's Proof, Formal Systems, and the Limits of Human Cognition

Cognitive Templates, Laws of Nature, and the Free Energy Principle

The Ghost in the Machine: Newton, Descartes, and the Limits of Intelligibility

Intelligible Theories, the Limits of Human Cognition, and the Beauty of Mystery

Reductionism, Holism, and the Limits of Scientific Explanation

Part 5: Connectionism vs. Symbolic Representation

Fodor and Feilichen's Critique of Connectionism: Productivity and Systematicity

Compositionality, Intention vs. Extension, and the Separation of Code and Memory

Abstraction, Training Problems, and the Need for New Approaches to Learning

Part 6: The Chomsky Interview: LLMs and Language

The Ethics of Deepfakes and the Recovery of the Chomsky Interview

Introduction to Noam Chomsky and the Futility of Large Language Models

Empiricism, Connectionism, and the Instinctive Nature of Radical Empiricism

Part 7: Intelligence, Biology, and Universal Grammar

Neuro-Symbolic Approaches, Engineering vs. Science, and the Limits of Silicon

Memory Models, Neural Nets, and the Search for the Right Lamppost

Cognitive Horizons, Prime Number Mazes, and the Limits of Human Understanding

Universal Grammar, the Language of Thought, and the Misunderstandings of Language

Part 8: Consciousness and Final Reflections

The Hard Problem of Consciousness and the Irrationality of Cognitive Science

Structure Dependence, Event Semantics, and the Mysteries of Language

Reflections on the Interview with Chomsky and the Miracle of Engineering

The ghost in the machine.

Machine Learning Street Talk

Part 1: Framing the Debate: LeCun vs. Chomsky

00:01Introduction to Noam Chomsky, Radical Empiricism, and the Antithesis of Jan LeCun

Introduction to Noam Chomsky, Radical Empiricism, and the Antithesis of Jan LeCun

02:48Yann LeCun's Path Towards Autonomous Machine Intelligence: A Critical Analysis

Yann LeCun's Path Towards Autonomous Machine Intelligence: A Critical Analysis

12:05The Limits of Perception-Derived Knowledge and the Need for Abstraction

The Limits of Perception-Derived Knowledge and the Need for Abstraction

Part 2: Deep Learning, Symbols, and Logic

17:23Compositionality, Symbolic Logic, and the Limitations of Deep Learning

Compositionality, Symbolic Logic, and the Limitations of Deep Learning

25:24Jürgen Schmidhuber's Critique of LeCun's Paper and the Debate Over Attribution

Jürgen Schmidhuber's Critique of LeCun's Paper and the Debate Over Attribution

36:31Energy-Based Models vs. Probability Theory: A Debate on Uncertainty Quantification

Energy-Based Models vs. Probability Theory: A Debate on Uncertainty Quantification

Part 3: Philosophical Foundations of AI

41:41World Models, Symbolic Systems, and the Empiricism vs. Rationalism Debate

World Models, Symbolic Systems, and the Empiricism vs. Rationalism Debate

53:41Empiricism, Rationalism, and the Role of Innate Knowledge in Cognition

Empiricism, Rationalism, and the Role of Innate Knowledge in Cognition

1:05:21Cognitive Templates, Surprise, and the Limits of Empiricism

Cognitive Templates, Surprise, and the Limits of Empiricism

Part 4: Limits of Cognition and Science

1:17:30Gödel's Proof, Formal Systems, and the Limits of Human Cognition

Gödel's Proof, Formal Systems, and the Limits of Human Cognition

1:25:30Cognitive Templates, Laws of Nature, and the Free Energy Principle

Cognitive Templates, Laws of Nature, and the Free Energy Principle

1:35:42The Ghost in the Machine: Newton, Descartes, and the Limits of Intelligibility

The Ghost in the Machine: Newton, Descartes, and the Limits of Intelligibility

1:41:48Intelligible Theories, the Limits of Human Cognition, and the Beauty of Mystery

Intelligible Theories, the Limits of Human Cognition, and the Beauty of Mystery

1:50:03Reductionism, Holism, and the Limits of Scientific Explanation

Reductionism, Holism, and the Limits of Scientific Explanation

Part 5: Connectionism vs. Symbolic Representation

2:00:04Fodor and Feilichen's Critique of Connectionism: Productivity and Systematicity

Fodor and Feilichen's Critique of Connectionism: Productivity and Systematicity

2:07:03Compositionality, Intention vs. Extension, and the Separation of Code and Memory

Compositionality, Intention vs. Extension, and the Separation of Code and Memory

2:17:02Abstraction, Training Problems, and the Need for New Approaches to Learning

Abstraction, Training Problems, and the Need for New Approaches to Learning

Part 6: The Chomsky Interview: LLMs and Language

2:27:03The Ethics of Deepfakes and the Recovery of the Chomsky Interview

The Ethics of Deepfakes and the Recovery of the Chomsky Interview

2:34:36Introduction to Noam Chomsky and the Futility of Large Language Models

Introduction to Noam Chomsky and the Futility of Large Language Models

2:38:44Empiricism, Connectionism, and the Instinctive Nature of Radical Empiricism

Empiricism, Connectionism, and the Instinctive Nature of Radical Empiricism

Part 7: Intelligence, Biology, and Universal Grammar

2:44:33Neuro-Symbolic Approaches, Engineering vs. Science, and the Limits of Silicon

Neuro-Symbolic Approaches, Engineering vs. Science, and the Limits of Silicon

2:50:52Memory Models, Neural Nets, and the Search for the Right Lamppost

Memory Models, Neural Nets, and the Search for the Right Lamppost

2:55:22Cognitive Horizons, Prime Number Mazes, and the Limits of Human Understanding

Cognitive Horizons, Prime Number Mazes, and the Limits of Human Understanding

3:06:32Universal Grammar, the Language of Thought, and the Misunderstandings of Language

Universal Grammar, the Language of Thought, and the Misunderstandings of Language

Part 8: Consciousness and Final Reflections

3:15:17The Hard Problem of Consciousness and the Irrationality of Cognitive Science

The Hard Problem of Consciousness and the Irrationality of Cognitive Science

3:24:00Structure Dependence, Event Semantics, and the Mysteries of Language

Structure Dependence, Event Semantics, and the Mysteries of Language

3:33:00Reflections on the Interview with Chomsky and the Miracle of Engineering

Reflections on the Interview with Chomsky and the Miracle of Engineering