Sequence Summary. This sequence documents research by SatisfIA, an ongoing project on non-maximizing, aspiration-based designs for AI agents that fulfill goals specified by constraints ("aspirations") rather than maximizing an objective function. We aim to contribute to AI safety by exploring design approaches and their software implementations that we believe might be promising but neglected or novel. Our approach is roughly related to but largely complementary to concepts like quantilization and satisficing (sometimes called "soft-optimization"), Decision Transformers, and Active Inference.

This post describes the purpose of the sequence, motivates the research, describes the project status, our working hypotheses and theoretical framework, and has a short glossary of terms. It does not contain results and can safely be skipped if you want to get directly into the actual research.

Epistemic status: We're still in the exploratory phase, and while the project has yielded some preliminary insights, we don't have any clear [...]

---

Outline:

(01:22) Purpose of the sequence

(02:18) Motivation

(02:37) (I) AGI development might be inevitable

(02:46) (II) It might be impossible to implement an objective function the maximization of which would be safe

(05:41) Project history and status

(07:34) Working hypotheses

(07:50) (III) We should not allow the maximization of any function of the state or trajectory of the world

(08:36) (IV) The core decision algorithm must be hard-coded

(09:30) (V) We should focus on model-based planning first and only consider learning later

(10:24) (VI) There are useful generic, abstract safety criteria largely unrelated to concrete human values

(11:22) (VII) One can and should provide certain guarantees on performance and behavior

(11:51) (VIII) Algorithms should be designed with tractability in mind, ideally employing Bellman-style recursive formulas

(12:45) Theoretical framework

(14:56) High-level structure of studied decision algorithms

(16:50) Appendix: Small glossary

(23:11) Acknowledgements

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
April 27th, 2024

Source:
https://www.lesswrong.com/posts/6BerZtxLQLgMSzA8n/aspiration-based-designs-1-informal-introduction

---

Narrated by TYPE III AUDIO.

“[Aspiration-based designs] 1. Informal introduction” by B Jacobs, Jobst Heitzig, Simon Fischer, Simon Dima

LessWrong (30+ Karma)