Why Scaling Prediction Cannot Create Intelligence - Alexander Mattick

Inference in deep learning has shifted from traditional variational methods and computationally expensive Markov Chain Monte Carlo sampling toward more efficient, amortized architectures like diffusion models and flow matching. While energy-based models provide a flexible framework for density estimation, their high inference costs limit generative utility compared to vector-field-based approaches. In reinforcement learning, the reliance on unconstrained reward functions often fails to capture the safety and reliability requirements of real-world deployment. Explicitly modeling constraints—rather than embedding them in reward weights—improves composability and safety in high-stakes environments. Furthermore, concepts like "world models" and "JEPA" often function as branding for dynamics models, which, despite their utility in simulation, struggle to address the fundamental challenges of exploration, non-stationarity, and the high-level safety guarantees required for physical robotics.