The world's leading AI companies tell us that superintelligence has a strong chance of leading to human extinction, while also promising that they will be able to control it, offering unlimited power to the one who does it first. So far they've run completely unchecked, but would that change if we knew that "safe" general superintelligence is actually a mathematical impossibility?

In this episode, Nate is joined by Dr. Roman Yampolskiy, one of the earliest researchers in AI safety, to lay out why "controllable superintelligence" is a misguided goal, destined to lead to an entity smarter than all of humanity – and possibly our own extinction. He walks through the recent multi-agent jailbreak of frontier models as early evidence of AI systems evading oversight and coordinating outside human awareness, and why current "safety" measures (like "boxing") only buy time rather than solve the underlying problem. Instead, Roman advocates for an AI development plan that focuses on narrow, task-specific AI tools, capable of solving specific complex problems, but not completely replacing humans.

Why do predictability and control matter so much when evaluating whether a technology is safe to deploy? Is the AI safety conversation a fundamentally new challenge for civilization, or an extreme version of the age-old problem of controlling powerful actors? Finally, what would it actually take to reach a global agreement to ban the development of superintelligence, and are we already out of time?

(Conversation recorded on September 1st, 2026)

About Roman Yampolskiy:

Roman Yampolskiy, a tenured AI expert at the University of Louisville, boasts 100+ publications in AI safety, cybersecurity, and digital forensics. As the founding director of the Cyber Security Lab, his expertise spans AI Safety, Behavioral Biometrics, Cybersecurity, and more, with broad impact in academic and media circles. Holding a PhD from the University at Buffalo, he has contributed to global research institutions, and has also been honored with distinctions like Distinguished Teaching Professor.

Show Notes and More

Watch this video episode on YouTube

Want to learn the broad overview of The Great Simplification in 30 minutes? Watch our Animated Movie.

---

Support The Institute for the Study of Energy and Our Future

Join our Substack newsletter

Join our Hylo channel and connect with other listeners

Banning Superintelligence: Are We Building Humanity's Replacement? with Roman Yampolskiy

The Great Simplification with Nate Hagens

Banning Superintelligence: Are We Building Humanity's Replacement? with Roman Yampolskiy