#335 Sriram Raghavan: Why IBM Is Betting Everything on Small AI Models

Small language models (SLMs) offer a more sustainable and efficient path for enterprise AI adoption than massive, monolithic models. By training 2-billion and 8-billion parameter models directly—rather than distilling them from larger ones—developers maintain base capabilities and safety alignment while controlling data lineage. Advanced techniques like reinforcement learning (RL) and inference-time scaling allow these compact models to match the performance of significantly larger systems on complex tasks like coding and mathematics. This shift toward "generative computing" treats models as modular components within a runtime, utilizing LoRa adapters and procedural logic to handle specific functions on demand. This architectural decomposition enables organizations to deploy fit-for-purpose AI across hybrid environments, optimizing for memory requirements and latency rather than raw parameter counts, ultimately facilitating faster innovation cycles and more reliable, governable agentic workflows.