Humans Need Entropy

The speaker discusses Karpathy's observation that humans, like AI models, tend to "collapse" over their lifespan, becoming less novel and more predictable as they age. This phenomenon, characterized by a decreasing learning rate and a shrinking "aperture" of interests, is a personal fear for the speaker. To counteract this, the speaker emphasizes the importance of seeking "sources of entropy" or novelty, similar to how children experience the world. He actively employs strategies like reading old books on rhetoric, engaging with Christopher Hitchens' works, and even developing AI tools to generate fresh language and assess content for "wows per minute" to maintain unpredictability and avoid intellectual stagnation. The speaker notes that while AI can exacerbate this issue by learning from "mediocrity," recognizing the problem is the first step for humans to continuously seek novelty and maintain a "young mind."