(If you’re in a hurry, you can just read the “Background and summary” section, and skip the other 85%.)

0. Background and summary

0.1 Background: What's the problem and why should we care?

My primary neuroscience research goal for the past couple years has been to solve a certain problem, a problem which has had me stumped since the very beginning of when I became interested in neuroscience at all (as a lens into Artificial General Intelligence safety) back in 2019. In this post I offer a hypothesis for what the solution might generally look like, at least in the big picture. I don’t have all the details pinned down, but I feel like this post is major progress. (Unless it's all wrong! Like the last one was.[1] Very happy for feedback!)

What is this grand problem? As described in Intro to Brain-Like AGI Safety, I believe the following:

[...]

---

Outline:

(00:13) 0. Background and summary

(00:18) 0.1 Background: What's the problem and why should we care?

(03:43) 0.2 Summary of the rest of the post

(08:10) 0.3 Confidence level

(08:52) 1. Ingredient 1: Innate sensory heuristics in the Steering Subsystem

(10:27) 1.1 Ingredient 1A: Innate sensory heuristics for conspecific detection in particular

(11:29) 1.2 Neuroscience details

(13:07) 2. Ingredient 2: Generalization via short-term predictors

(16:09) 2.1 Neuroscience details

(17:23) 3. Ingredient 3: Tailoring learned models via involuntary attention and learning rate

(17:43) 3.1 What does the orienting reflex do?

(20:30) 3.1.1 Side note: Transient attentional gaps are more common, and harder to notice, than you realize

(22:17) 3.2 Combining attention with time-variable learning rates

(25:25) 3.3 Neuroscience details

(26:33) 4. Ingredient 4: Reading out transient empathetic simulations

(30:18) 4.1 So, the “thinking of a conspecific” flag is also a “this is an empathetic simulation” flag?

(33:10) 4.2 Neuroscience details

(34:42) 5. Hypothesis: a “compassion / spite circuit”

(35:04) 5.1 The “Conspecific seems to be feeling (dis)pleasure” signal

(36:36) 5.2 The “friend (+) vs enemy (–)” parameter

(39:13) 5.2.1 Evolution and zoological context

(41:49) 5.2.2 Neuroscience details

(42:19) 5.3 Phasic physiological arousal

(43:41) 5.3.1 Neuroscience details

(45:22) 5.4 Generalization via short-term predictors

(45:53) 6. The “compassion / spite circuit” also causes a “drive to feel liked / admired”

(46:49) 6.1 Key idea: My “compassion / spite circuit” is disproportionately active and important while the conspecific is thinking about me-in-particular

(47:00) 6.1.1 Starting example: Innate sensory heuristics for receiving eye contact

(48:22) 6.1.2 Generalization: Innate sensory heuristics fire strongly upon being the target of an orienting reflex

(49:18) 6.1.3 Another example: Somebody deliberately getting my attention

(49:57) 6.2 If the same circuit drives both compassion and “drive to feel liked / admired”, why aren’t they more tightly correlated across the population?