As part of our Summer 2024 Program, MATS ran a series of discussion groups focused on questions and topics we believe are relevant to prioritizing research into AI safety. Each weekly session focused on one overarching question, and was accompanied by readings and suggested discussion questions. The purpose of running these discussions was to increase scholars’ knowledge about the AI safety ecosystem and models of how AI could cause a catastrophe, and hone scholars’ ability to think critically about threat models—ultimately, in service of helping scholars become excellent researchers.

The readings and questions were largely based on the curriculum from the Winter 2023-24 Program, with two changes:

We reduced the number of weeks, since in the previous cohort scholars found it harder to devote time to discussion groups later in the program.
For each week we selected a small number of “core readings”, since many scholars were unable [...]

---

Outline:

(01:57) Week 1: How powerful is intelligence?

(02:03) Core readings

(02:37) Other readings

(03:34) Discussion questions

(04:50) Week 2: How and when will transformative AI be made?

(04:56) Core readings

(07:06) Other readings

(09:23) Discussion questions

(10:30) Week 3: How could we train AIs whose outputs we can’t evaluate?

(10:37) Core readings

(12:00) Other readings

(15:01) Discussion questions

(16:01) Week 4: Will AIs fake alignment?

(16:06) Core readings

(16:25) Other readings

(16:28) On inner and outer alignment

(17:15) On reasons to think deceptive alignment is likely

(18:07) Discussion questions

(19:25) Week 5: How should AI be governed?

(19:31) Core readings

(20:49) Other readings

(24:10) Discussion questions

(25:32) Readings that did not fit into any specific week

(26:26) Acknowledgements

---

First published:
October 7th, 2024

Source:
https://www.lesswrong.com/posts/rhEXTkDmssrHBNrfS/mats-ai-safety-strategy-curriculum-v2

---

Narrated by TYPE III AUDIO.

“MATS AI Safety Strategy Curriculum v2” by DanielFilan, Ryan Kidd

LessWrong (30+ Karma)