Summary

This post gives my personal take on “AI for epistemics” and how important it might be to work on.

Some background context:

AI capabilities are advancing rapidly and I think it's important to think ahead and prepare for the possible development of AI that could automate almost all economically relevant tasks that humans can do.[1]
That kind of AI would have a huge impact on key epistemic processes in our society. (I.e.: It would have a huge impact on how new facts get found, how new research gets done, how new forecasts get made, and how all kinds of information spread through society.)
I think it's very important for our society to have excellent epistemic processes. (I.e.: For important decisions in our society to be made by people or AI systems who have informed and unbiased beliefs that take into account as much of the available evidence [...]

---

Outline:

(00:04) Summary

(04:36) Structure of the post

(05:33) Previous work

(07:29) Why work on AI for epistemics?

(07:34) Summary

(09:38) Importance

(12:15) Path-dependence

(14:45) To be more concrete

(15:37) Good norms and practices for AI-as-knowledge-producers

(17:12) Good norms and practices for AI-as-communicators

(19:30) Differentially high epistemic capabilities

(27:12) Heuristics for good interventions

(28:01) Direct vs. indirect strategies

(29:24) Indirect value generation

(33:30) On long-lasting differential capability improvements

(36:11) Painting a picture of the future

(41:04) Concrete projects for differentially advancing epistemic capabilities

(43:47) Evals/benchmarks for forecasting (or other ambitious epistemic assistance)

(44:27) Automate forecasting question-generation and -resolution

(45:59) Logistics of past-casting

(50:22) Start efforts inside of AI companies for AI forecasting or other ambitious epistemic assistance

(51:34) Scalable oversight / weak-to-strong-generalization / ELK

(53:53) Experiments on what type of arguments and AI interactions tend to lead humans toward truth vs. mislead them

(56:12) Concluding thoughts

The original text contained 22 footnotes which were omitted from this narration.

---

First published:
August 24th, 2024

Source:
https://www.lesswrong.com/posts/D2n5uduYGXuexkv7v/what-s-important-in-ai-for-epistemics-2

---

Narrated by TYPE III AUDIO.

“What’s important in ‘AI for epistemics’? ” by Lukas Finnveden

LessWrong (30+ Karma)