What Comes After Transformers?

AI development is shifting toward post-transformer architectures that prioritize latent reasoning and continual learning over massive, static parameter scaling. Zuzanna Stamirovska, CEO of Pathway, argues that current transformer-based models are inefficient for reasoning and prone to catastrophic forgetting. By utilizing synaptic plasticity and "fast weights," Pathway’s models achieve significant reasoning capabilities at a fraction of the compute cost, as demonstrated by their performance on the ArcAGI benchmark. These "Small Reasoning Models" (SRMs) offer a path toward autonomous, data-efficient systems capable of real-time adaptation and regime-change detection. This approach challenges the necessity of massive foundation models, suggesting that specialized, smaller architectures can deliver superior performance for enterprise and edge-computing applications while maintaining cost-efficiency and operational control.