Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

This is a linkpost for https://www.apolloresearch.ai/blog/we-need-a-science-of-evals

In this post, we argue that if AI model evaluations (evals) want to have meaningful real-world impact, we need a “Science of Evals”, i.e. the field needs rigorous scientific processes that provide more confidence in evals methodology and results.

Model evaluations allow us to reduce uncertainty about properties of Neural Networks and thereby inform safety-related decisions. For example, evals underpin many Responsible Scaling Policies and future laws might directly link risk thresholds to specific evals. Thus, we need to ensure that we accurately measure the targeted property and we can trust the results from model evaluations. This is particularly important when a decision not to deploy the AI system could lead to significant financial implications for AI companies, e.g. when these companies then fight these decisions in court.

---

Outline:

(04:44) What do we mean by “Science of Evals”?

(06:37) Maturation process of a field

(09:38) Current work in the direction of Science of Evals

(13:14) Next steps

(13:24) Field building

(15:20) Open research questions

(18:14) Conclusion

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
January 22nd, 2024

Source:
https://www.lesswrong.com/posts/fnc6Sgt3CGCdFmmgX/we-need-a-science-of-evals

---

Narrated by TYPE III AUDIO.

“We need a science of evals” by Marius Hobbhahn, Jérémy Scheurer

LessWrong (30+ Karma)