DPO Meets PPO: Reinforced Token Optimization for RLHF

Arxiv Papers - DPO Meets PPO: Reinforced Token Optimization for RLHF