Introduction and summary

(This is the fourth in a series of four posts about how we might solve the alignment problem. See the first post for an introduction to this project and a summary of the series as a whole.)

In the first post in this series, I distinguished between “motivation control” (trying to ensure that a superintelligent AI's motivations have specific properties) and “option control” (trying to ensure that a superintelligent AI's options have specific properties). My second post surveyed some of the key challenges and available approaches for motivation control; and my third post did the same for option control.

In this fourth and final post, I wrap up the series with a discussion of two final topics: namely, moving from motivation/option control to safety-from-takeover (let's call this “incentive design”), and capability elicitation. Here's a summary: