Imitation learning vs. offline reinforcement learning

Sergey Levine delivers a lecture comparing imitation learning and reinforcement learning, particularly in offline settings. He addresses whether behavioral cloning or offline RL should be used with near-optimal data, and whether behavioral cloning can solve RL problems. Levine suggests offline RL is generally preferable, even with optimal data, due to its ability to handle critical states and benefit from slightly suboptimal data that improves coverage. He also notes that while behavior cloning can address RL problems, it requires careful inductive bias. Combining behavior cloning with planning can yield effective offline RL methods, as illustrated by trajectory transformers, deep imitative models, and Viking, which all leverage a density model and a planning procedure.