EP17: RL with Will Brown

In this episode of the Information and Bottleneck podcast, Will Brown, an expert in reinforcement learning (RL), discusses his background in math, computer science, and philosophy, leading to his work in algorithmic game theory and multi-agent reinforcement learning. He explains his transition from theoretical research to practical applications, particularly in large language models (LLMs). The conversation covers the nuances of RL in LLMs, distinguishing between true online RL and offline optimization, and explores the importance of problem selection, reward models, and the potential of agentic tool use, search, and coding. Will shares insights on the components needed for effective RL, the relevance of historical RL techniques, and the future of RL training pipelines, including the role of mid-training and synthetic data. He also touches on the open-source initiatives at Prime Intellect, such as the Environments Hub and Verifiers library, aimed at facilitating RL research and development.