Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

At some point in the future, AI developers will need to ensure that when they train sufficiently capable models, the weights of these models do not leave the developer's control. Ensuring that weights are not exfiltrated seems crucial for preventing threat models related to both misalignment and misuse. The challenge of defending model weights has previously been discussed in a RAND report.

In this post, I’ll discuss a point related to preventing weight exfiltration that I think is important and under-discussed: unlike most other cases where a defender wants to secure data (e.g. emails of dissidents or source code), model weights are very large files. At the most extreme, it might be possible to set a limit on the total amount of data uploaded from your inference servers so that an attacker would be unable to exfiltrate [...]

---

Outline:

(03:30) Summary of key considerations

(04:48) Epistemic status

(06:25) More detailed arguments

(06:29) The total amount of generated tokens is probably not much larger than the number of model parameters (for an AI lab's best model)

(08:37) We can probably heavily compress (text) model outputs

(14:07) Other approaches for limiting uploads

(16:33) Tracking and restricting total uploads seems relatively doable from a security perspective

(21:05) Even if total compressed uploads are larger than model weights, being within a moderate factor might make exfiltration via uploads hard

(22:45) Remaining concerns

(24:24) Appendix: experiment with compressing T=0 generations from weak LLMs

---

First published:
February 6th, 2024

Source:
https://www.lesswrong.com/posts/rf66R4YsrCHgWx9RG/preventing-exfiltration-via-upload-limits-seems-promising

---

Narrated by TYPE III AUDIO.

“Preventing exfiltration via upload limits seems promising” by ryan_greenblatt

LessWrong (30+ Karma)