Ep. 158: abstraction and agency | John Wentworth

The podcast explores the concepts of agency and abstraction in the context of AI, featuring John Wentworth, an independent AI researcher. Wentworth discusses his research on abstraction, questioning how minds identify natural objects and converge on similar concepts. He touches on the mechanics of being an independent researcher, including funding through grants and the shift from theoretical work to product development, such as an AI image editor that edits images in concept space. The conversation covers different levels of agency, distinguishing between systems optimized to do something and those actively planning. Wentworth also addresses AI alignment, emphasizing the importance of shared concepts between humans and AI to solve the pointer problem and ensure AI values align with human values.