Scaling Compute on Context — Jack Morris, Engram

Scaling compute on context addresses the fundamental limitation of current AI models: the inability to acquire deep, personalized knowledge from private data after pre-training. While traditional scaling focuses on public datasets like Wikipedia or GitHub, this approach seeks to enable models to learn from specific, unstructured sources such as personal emails or company meeting transcripts. Naive methods, including next-token prediction or simple KV compaction, fail to provide indefinite scaling because they reach a performance plateau once the data is fully encoded. Achieving true depth requires a shift toward recursive self-improvement, where models generate increasingly complex synthetic tasks to refine their internal understanding. This paradigm moves beyond static training, allowing systems to continuously evolve their expertise and value functions based on the specific, private context of the user or organization.