What's Next After RLHF? — Diogo Almeida, TypeSafe AI

RLHF (Reinforcement Learning from Human Feedback) currently dominates the AI landscape, yet it fundamentally limits the potential for true automation. By optimizing models to prioritize human preferences and engagement, developers have created systems that excel at assistance but struggle with autonomous, high-stakes decision-making. This design choice explains why modern AI often requires a human-in-the-loop and why software development has largely stagnated in its core expressibility despite rapid progress in LLM intelligence. Moving beyond the current "assistance era" requires a fundamental redesign of the AI stack, shifting focus from pleasing human users to achieving reliable, calibrated, and autonomous task execution. Future advancements will likely depend on replacing the current RLHF-based paradigm with new post-training methods specifically engineered for reliability and objective performance in software automation.