18:I[66739,["/_next/static/chunks/0u0nyy7wb9me2.js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i","/_next/static/chunks/05hx6kuf6-oy4.js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i","/_next/static/chunks/12hmyxte1w1dw.js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i","/_next/static/chunks/166.u_60qj09..js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i","/_next/static/chunks/0g9_wl34k-6l4.js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i","/_next/static/chunks/06dn~uer6.43l.js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i"],"default"] 1a:I[38632,["/_next/static/chunks/0qqtp3r0h7kah.js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i","/_next/static/chunks/0iukvycbd42rl.js?dpl=dpl_EVxQmm5ee6SfSwpTLEi7dkcqj48i"],"IconMark"] 19:T4f8,

This paper introduces Tail-Likelihood Reinforcement Learning (TailRL), a novel optimization framework designed to improve how generative policies handle continuous rewards. Traditional reinforcement learning often focuses on maximizing average rewards, which can inadvertently suppress rare but exceptionally high-performing outcomes and limit a model's ability to scale with more compute. TailRL addresses this by maximizing the log-probability of exceeding diverse reward thresholds, effectively treating a continuous signal as a collection of binary success events. This approach places greater mathematical weight on the upper tail of the reward distribution, ensuring that infrequent, high-quality samples are prioritized during training. Empirical tests across tasks like maze navigation and code optimizationdemonstrate that TailRL prevents suboptimal collapse and significantly boosts performance during inference-time sampling. Ultimately, the method provides a simple, critic-free way to align policy training with the goal of finding the best possible solutions rather than just the most common ones.

Tail-Likelihood Reinforcement Learning

Best AI papers explained

Tail-Likelihood Reinforcement Learning