IMO it is harmful on expectation for a technical safety researcher to work at DeepMind, OpenAI or Anthropic.

Four reasons:

Interactive complexity. The intractability of catching up – by trying to invent general methods for AI corporations to somehow safely contain model interactions, as other engineers scale models' combinatorial complexity and outside connectivity.
Safety-capability entanglements
1. Commercialisation. Model inspection and alignment techniques can support engineering and productisation of more generally useful automated systems.
2. Infohazards. Researching capability risks within an AI lab can inspire researchers who hear about it to use your findings to build in capabilities.
Shifts under competitive pressure
1. DeepMind merged with Google Brain to do commercialisable research[1], OpenAI set up a company and partnered with Microsoft to release ChatGPT,
  Anthropic pitched to investors they'd build a model 10 times more capable.
2. If you are an employee at one of these for-profit corporations [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
February 7th, 2024

Source:
https://www.lesswrong.com/posts/Foh24Ya82bhpGRWME/why-i-think-it-s-net-harmful-to-do-technical-safety-research

---

“Why I think it’s net harmful to do technical safety research at AGI labs” by Remmelt

LessWrong (30+ Karma)