Geoffrey Hinton | On working with Ilya, choosing problems, and the power of intuition | Sana

This podcast episode explores the early experiences and influences that shaped the speaker's understanding of computer science and AI. It delves into the concept of hidden layers in neural nets, the role of predicting the next symbol in language models, the learning capabilities of big language models, the improvement of reasoning abilities as models scale up, the different views of language and cognition, the relationship between GPUs and computation, the power of symbol processing, the importance of backpropagation, the potential applications of AI in healthcare, and the significance of intuition and learning algorithms in research.

Outlines

Sign in to continue reading, translating and more.

Continue

Geoffrey Hinton | On working with Ilya, choosing problems, and the power of intuition

Sana

The Beginning of Geoff's Journey into AI and Computer Science

The Journey of Neural Nets and the Question of Weight Modification

Hidden Markov Models, Backpropagation, and Collaboration in Neural Nets

The Role of Predicting the Next Symbol in Understanding Language Models

Big Language Models and Creative Learning

Models, Reasoning, and Multimodality: Scaling Up for Better Understanding

The Three Views of Language and Cognition: Symbolic, Vector, and Embedding Approaches

The Evolution of GPUs and the Challenge of Fast Weights in Neural Networks

The Impact of Models on Learning and the Misconception of Innate Knowledge

Selecting Problems and the Power of Symbol Processing

Does the Brain Do Backpropagation? A Critical Question in Neuroscience

The Impacts of AI on Healthcare and Concerns about Bad Actors

The Importance of Intuition and Learning Algorithms in Research

Geoffrey Hinton | On working with Ilya, choosing problems, and the power of intuition

Sana

00:00The Beginning of Geoff's Journey into AI and Computer Science

The Beginning of Geoff's Journey into AI and Computer Science

01:59The Journey of Neural Nets and the Question of Weight Modification

The Journey of Neural Nets and the Question of Weight Modification

05:05Hidden Markov Models, Backpropagation, and Collaboration in Neural Nets

Hidden Markov Models, Backpropagation, and Collaboration in Neural Nets

08:56The Role of Predicting the Next Symbol in Understanding Language Models

The Role of Predicting the Next Symbol in Understanding Language Models

11:21Big Language Models and Creative Learning

Big Language Models and Creative Learning

15:13Models, Reasoning, and Multimodality: Scaling Up for Better Understanding

Models, Reasoning, and Multimodality: Scaling Up for Better Understanding

19:16The Three Views of Language and Cognition: Symbolic, Vector, and Embedding Approaches

The Three Views of Language and Cognition: Symbolic, Vector, and Embedding Approaches

23:12The Evolution of GPUs and the Challenge of Fast Weights in Neural Networks

The Evolution of GPUs and the Challenge of Fast Weights in Neural Networks

27:20The Impact of Models on Learning and the Misconception of Innate Knowledge

The Impact of Models on Learning and the Misconception of Innate Knowledge

31:31Selecting Problems and the Power of Symbol Processing

Selecting Problems and the Power of Symbol Processing

35:06Does the Brain Do Backpropagation? A Critical Question in Neuroscience

Does the Brain Do Backpropagation? A Critical Question in Neuroscience

37:51The Impacts of AI on Healthcare and Concerns about Bad Actors

The Impacts of AI on Healthcare and Concerns about Bad Actors

41:48The Importance of Intuition and Learning Algorithms in Research

The Importance of Intuition and Learning Algorithms in Research