In this post, I discuss a number of critiques of Guaranteed Safe AI (GSAI) approaches I’ve encountered since releasing this position paper.

The critiques I’m responding to here are taken from a range of places, including personal conversations and social media. The two most substantive written/recorded critiques I'm aware of (and had in mind when writing this post) are Andrew Dickson's Limitations on Formal Verification for AI Safety and Zac Hatfield-Dodds's talk Formal Verification is Overrated. I’ve done my best to do justice in our depiction of those critiques, but I certainly welcome clarifications or pushback where I seem to have missed the original point of a given critique.

To be clear, the position I'm defending here (and will be adding more nuance to throughout this piece) is not that GSAI is definitely going to succeed. Nor that it is a solution to all of AI risk. What [...]

---

Outline:

(03:21) Acknowledgements

(03:43) 1. Brief summary of Towards Guaranteed Safe AI

(10:05) 2. Commonly-encountered critiques, and some responses

(10:17) 1. The proposed architecture won't work

(10:22) 1.1. You can't get a good enough world model

(19:42) 1.2. You can't get good enough guarantees

(26:43) 1.3. You can't get good enough safety specifications

(30:14) 2. Other approaches are a better use of time

(30:19) 2.1. Does not address the most serious risks

(44:04) 2.2. AI won't help (soon) enough

(48:20) 2.3. Lack of 'portability and independent verification of proof certificate

(49:48) 3. Conclusion

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
January 31st, 2025

Source:
https://www.lesswrong.com/posts/DZuBHHKao6jsDDreH/in-response-to-critiques-of-guaranteed-safe-ai

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“In response to critiques of Guaranteed Safe AI” by Nora_Ammann

LessWrong (30+ Karma)