Thomas Akam on Model-based RL in the Brain

In this episode of the Talk RL Podcast, host Robin Chohan interviews Professor Thomas Akam, a neuroscientist at Oxford University, about the intersection of reinforcement learning (RL) and the brain. Professor Akam discusses his research on how the brain generates flexible behavior through internal models and learning algorithms. The conversation covers the mapping of computational RL to brain circuits, particularly the role of dopamine neurons and the basal ganglia, and explores the complexities of model-free versus model-based RL. Professor Akam also shares insights on how time is modeled in the brain, challenges to the canonical theory of dopamine and reward prediction error, and the efficiency of the human brain compared to computational AI. He touches on current research using complex mazes to understand how the brain plans action sequences and the rapid advancements in neuroscience tools like Neuropixels probes and optogenetics.