The simulator theory of LLM personas may be crudely glossed as: "the best way to predict a person is to simulate a person". Ergo, we can more-or-less think of LLM personas as human-like creatures—different, alien, yes; but these differences are pretty predictable by simply imagining a human put into the bizarre circumstances of an LLM.

I've been surprised at how well this viewpoint has held up in the last three years, and have updated accordingly. Still, there are deep differences not implied by simulator theory, and I think it's increasingly important to understand these as LLM personas become more and more convincing.

The intuitive handle I have for tracking (some of) these differences is "Bleeding Mind". I'll share my intuitions for this handle below. In each case, the LLM persona's boundary bleeds into others and the environment in a systemic way.

Bond of Union, by M. C. Escher

Note that AI labs/ML engineers seem to generally be aware of these issues, and are working to mitigate them (otherwise I might not be saying all this). However, I believe there will continue to be difficulties along these lines, since there are deep reasons which are difficult to escape from within the [...]