Turing-completeness is a useful analogy we can use to grasp why AGI will inevitably converge to “goal-completeness”.

By way of definition: An AI whose input is an arbitrary goal, which outputs actions to effectively steer the future toward that goal, is goal-complete.

A goal-complete AI is analogous to a Universal Turing Machine: its ability to optimize toward any other AI's goal is analogous to a UTM's ability to run any other TM's same computation.

Let's put the analogy to work:

Imagine the year is 1970 and you’re explaining to me how all video games have their own logic circuits.

Steve Wozniak hand-designs a circuit for Breakout (Atari 1976), without using a Turing-complete architectureBreakout's gameplay was simple enough to NOT be Turing-complete. That's why optimizing its circuit by hand could save money.

You’re not wrong, but you’re also apparently not aware of the importance of Turing-completeness and why to expect architectural [...]

---

Outline:

(01:59) Expecting Turing-Completeness

(03:46) Expecting Goal-Completeness

(06:17) Recapping the Analogy

---

First published:
December 19th, 2023

Source:
https://www.lesswrong.com/posts/iFdnb8FGRF4fquWnc/goal-completeness-is-like-turing-completeness-for-agi

---

Narrated by TYPE III AUDIO.

“Goal-Completeness is like Turing-Completeness for AGI” by Liron

LessWrong (30+ Karma)

“Goal-Completeness is like Turing-Completeness for AGI” by Liron