Human Experience Is the Bottleneck on AI | Rudolf Laine

Successionism—the prospect of AI assuming humanity’s dominant role—poses fundamental questions about the future of agency and moral judgment. Current discourse often reflects a Benthamite focus on maximizing value or pleasure, which risks treating humans as replaceable components in an optimized system. In contrast, a Millian framework centers on the development of autonomous individuals, arguing that human experience is the essential, non-automatable ground-truth for moral progress. Because moral innovation relies on subjective human experience rather than purely rational deduction, humans remain a necessary bottleneck in the automation of philosophy. Rudolf Laine, a member of technical staff at Thinking Machines and author of the *No Set Gauge* substack, highlights that treating AI as a successor requires careful navigation of these philosophical traditions to ensure that future developments support human freedom and individual growth rather than mere utility maximization.