17:I[66739,["/_next/static/chunks/0.3dg.jzl0xwf.js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi","/_next/static/chunks/05hx6kuf6-oy4.js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi","/_next/static/chunks/0e~mn1op2jzz5.js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi","/_next/static/chunks/0juapqyy99ga..js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi","/_next/static/chunks/09-r7sdqv1jf4.js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi","/_next/static/chunks/06dn~uer6.43l.js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi"],"default"] 19:I[38632,["/_next/static/chunks/0qqtp3r0h7kah.js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi","/_next/static/chunks/0iukvycbd42rl.js?dpl=dpl_ADavNnop8X1LgnurSAwpmKs3ySqi"],"IconMark"] 18:T47b,

This paper introduces a novel reinforcement learning framework called Actor-Critic without Actor (ACA), which is designed to be a lightweight and efficient alternative to traditional actor-critic methods. ACA eliminates the explicit actor network, generating actions instead from the gradient field of a noise-level critic via a diffusion-based denoising process. This method significantly reduces algorithmic and computational overhead compared to standard and diffusion-based actor-critic approaches, as demonstrated by requiring substantially fewer parameters and achieving competitive performance on online RL benchmarks like MuJoCo tasks. A key feature of ACA is its noise-level critic, which conditions value estimates on the diffusion timestep, stabilizing gradients and ensuring the policy maintains immediate alignment with the critic's latest value updates while preserving multi-modal action coverage. Overall, ACA offers a simplified, expressive, and parameter-efficient solution for online reinforcement learning.

Actor-Critic without Actor: Critic-Guided Denoising for RL

Best AI papers explained

Actor-Critic without Actor: Critic-Guided Denoising for RL