Andrej Karpathy — “We’re summoning ghosts, not building animals”

In this interview, Andrej Karpathy discusses his perspectives on the progress and future of AI agents, emphasizing that it's the "decade of agents" rather than just a year due to the significant work still needed in areas like continual learning, multimodality, and computer use. He reflects on the history of AI, including the missteps of early reinforcement learning and the importance of representation learning with LLMs. Karpathy also touches on the differences between building AI and the evolution of animal intelligence, the role of pre-training as "crappy evolution," and the need to remove knowledge from AI models to enhance their cognitive core. He further explores the limitations of current RL methods, the potential of process-based supervision, and the challenges of model collapse, and shares his insights on the future of AI architecture, the value of coding models, and the importance of education in empowering humanity in an increasingly automated world.