Memory and Continual Learning: Engram's Dan Biderman and Jessy Lin

AI models currently face a bottleneck in understanding evolving, private, or company-specific contexts, often relying on inefficient retrieval-augmented generation or massive context windows. Engram co-founders Dan Biderman and Jessy Lin propose a shift toward "always training" models that internalize domain-specific knowledge directly into their weights. By utilizing adapter fine-tuning techniques like LoRA, these models achieve superior performance and efficiency, potentially reducing token inference consumption by orders of magnitude compared to traditional retrieval methods. This approach treats memory and continual learning as fundamental components of intelligence, arguing that internalizing context—rather than just retrieving it—enables models to form the abstract associations necessary for complex reasoning. Ultimately, this paradigm aims to move beyond generic frontier models, allowing individuals and organizations to develop bespoke, continuously improving AI agents that adapt to their unique workflows and priorities.