Audio note: this article contains 42 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

I believe that the theoretical foundations of the AIXI agent and variations are a surprisingly neglected and high leverage approach to agent foundations research. Though discussion of AIXI is pretty ubiquitous in A.I. safety spaces, underscoring AIXI's usefulness as a model of superintelligence, this is usually limited to poorly justified verbal claims about its behavior which are sometimes questionable or wrong. This includes, in my opinion, a serious exaggeration of AIXI's flaws. For instance, in a recent post I proposed a simple extension of AIXI off-policy that seems to solve the anvil problem in practice - in fact, in my opinion it has never been convincingly argued that the anvil problem would occur for an AIXI approximation. The [...]

---

Outline:

(04:46) Computability Foundations

(06:35) Semimeasure Theory

(08:58) Algorithmic Probability Foundations

(10:58) AIXI Generalizations

(14:40) Scaffolding the Universal Distribution

(18:25) Embedded Versions of AIXI

The original text contained 6 footnotes which were omitted from this narration.

---

First published:
September 12th, 2024

Source:
https://www.lesswrong.com/posts/MvfD4tmzyuCYFqB2f/open-problems-in-aixi-agent-foundations

---

Narrated by TYPE III AUDIO.

“Open Problems in AIXI Agent Foundations ” by Cole Wyeth

LessWrong (30+ Karma)