[NeurIPS Best Paper] 1000 Layer Networks for Self-Supervised RL — Kevin Wang et al, Princeton

In this episode of Latent Space, the host interviews Kevin Wang and his team (Benjamin Eysenbach, and Ishaan) from Princeton about their NeurIPS best paper award-winning project on scaling deep reinforcement learning (RL). Kevin discusses the motivation behind exploring deeper networks in RL, drawing parallels to the success of large models in NLP and vision. The team explains their approach of using self-supervised RL with architectural innovations like residual connections and layer normalization to achieve significant performance gains. They also touch on the intersection of reinforcement learning and self-supervised learning, the potential impact on robotics, and the trade-offs involved in scaling depth versus width in neural networks. The team also discusses future research directions, including stitching in reinforcement learning, scaling up depth, width and batch size, and vision language action models.