This is a snapshot of a new page on the AI Impacts Wiki.

We’ve made a list of arguments1 that AI poses an existential risk to humanity. These are intended as summaries—they may not contain the necessary detail to be compelling, or to satisfyingly represent the position. We’d love to hear how you feel about them in the comments and polls.

Competent non-aligned agents

Humans increasingly lose games to the best AI systems. If AI systems become similarly adept at navigating the real world, will humans also lose out? (Image: Midjourney)

Summary:

Humans will build AI systems that are 'agents', i.e. they will autonomously pursue goals
Humans won’t figure out how to make systems with goals that are compatible with human welfare and realizing human values
Such systems will be built or selected to be highly competent, and so gain the power to achieve their [...]