Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

This is a linkpost for https://www.apolloresearch.ai/blog/a-starter-guide-for-evals

This is a starter guide for model evaluations (evals). Our goal is to provide a general overview of what evals are, what skills are helpful for evaluators, potential career trajectories, and possible ways to start in the field of evals.

Evals is a nascent field, so many of the following recommendations might change quickly and should be seen as our current best guess.

Why work on evals?

Model evaluations increase our knowledge about the capabilities, tendencies, and flaws of AI systems. Evals inform the public, AI organizations, lawmakers, and others and thereby improve their decision-making. However, similar to testing in a pandemic or pen-testing in cybersecurity, evals are not sufficient, i.e. they don’t increase the safety of the model on their own but are needed for good decision-making and [...]

---

Outline:

(00:37) Why work on evals?

(01:39) What are model evaluations (evals)?

(05:11) What skills are helpful for evaluators?

(05:32) LLM steering

(06:09) Prompting

(07:59) Supervised fine-tuning (SFT)

(09:02) RL with LLMs

(09:45) Scaffolding and LM agents

(10:33) Tool use

(10:51) Generalist

(12:52) Scientific mindset

(13:54) Empirical research experience

(15:57) Software engineering

(16:42) Potential career paths

(17:16) Engineering-focused

(19:06) Research-focused

(20:17) How to start with evals work?

(20:27) Hands-on approach

(21:42) Learning general skills approach

(22:58) Contributions

---

First published:
January 8th, 2024

Source:
https://www.lesswrong.com/posts/2PiawPFJeyCQGcwXG/a-starter-guide-for-evals

---

Narrated by TYPE III AUDIO.

“A starter guide for evals” by Marius Hobbhahn, Mikita Balesni, Jérémy Scheurer, rusheb

LessWrong (30+ Karma)