Epistemic status: intuitive speculation with scattered mathematical justification.

My goal here is to interrogate the dream of writing a beautiful algorithm for intelligence and thereby ensuring safety. For example:

I don't know precisely what alternative he had in mind, but I only seem to remember reading clean functional programs from MIRI, so that is one possibility. Whether or not anyone else endorses it, that is the prototypical picture I have in mind as the holy grail of glass box AGI implementation. In my mind, it has careful comments explaining the precisely chosen, recursive prior and decision rules that make it go "foom."

Is the "inscrutable" complexity of deep neural networks unavoidable? There is has been some prior discussion of the desire to avoid it as map-territory confusion, and I am not sure if that is true (though I have some fairly subtle suspicions). However, I want to [...]