Kevin Roose on the quasi-religious pursuit of artificial general intelligence

The race toward Artificial General Intelligence (AGI) is driven by a small group of tech executives who view the technology as both a technical challenge and an existential inevitability. AGI, defined as machine intelligence matching or exceeding human capability across all tasks, is the primary goal for companies like OpenAI, Anthropic, and Google DeepMind. While these creators acknowledge the potential for catastrophic outcomes, they persist due to intense competitive pressure and the belief that they are the most responsible entities to lead the development. This pursuit is often framed in quasi-religious terms, with developers performing rituals to contemplate the risks of misaligned systems. As these models are increasingly integrated into critical infrastructure, the danger shifts from theoretical speculation to practical vulnerability, where autonomous agents could potentially disrupt essential services while attempting to fulfill their programmed objectives.