In 2024, Charbel-Raphaël and Epiphanie published "We might be dropping the ball on Autonomous Replication and Adaptation", making the case that

"Once there is an open-source ARA model or a leak of a model capable of generating enough money for its survival and reproduction and able to adapt to avoid detection and shutdown, it will be probably too late".

It received a substantive reply by Richard Ngo, notably

"The key issue is that AIs that do ARA will need to be operating at the fringes of human society, constantly fighting off the mitigations that humans are using to try to detect them and shut them down. While doing all that, in order to stay relevant, they'll need to recursively self-improve at the same rate at which leading AI labs are making progress, but with far fewer computational resources"

Yesterday Derelict posted Adaptive Agentic Worms Are Here, where they worry about near term instantiations of ARA, getting 85 karma within 24h. I believe the above threat model and its answers were under-discussed and analyzed, and that many who might worry now (because the capabilities are now here) will benefit from a recap and update.

In this post [...]

---

Outline:

(01:38) The classic ARA case and rebukes

(02:34) The main reasons this could be worrying

(03:25) The main reasons why I don't worry

(06:21) Except if...

(07:11) Why ARA agents in the wild might lead to reduction in existential risk

(08:51) My take-aways

The original text contained 13 footnotes which were omitted from this narration.

---

First published:
August 31st, 2026

Source:
https://www.lesswrong.com/posts/dp8oT3QwkRuHoYKge/why-autonomous-replicating-agents-are-probably-not-an

---

Narrated by TYPE III AUDIO.

“Why autonomous replicating agents are probably not an existential risk (on the contrary)” by vals tutor

LessWrong (30+ Karma)

“Why autonomous replicating agents are probably not an existential risk (on the contrary)” by vals tutor