Or rather, we don’t actually have a proper o1 system card, aside from the outside red teaming reports. At all.

Because, as I realized after writing my first draft of this, the data here does not reflect the o1 model they released, or o1 pro?

I think what happened is pretty bad on multiple levels.

The failure to properly communicate the information they did provide.
The failure to provide the correct information.
The failure, potentially, to actually test the same model they released, in many of the ways we are counting on to ensure the model is safe to release.
The failure to properly elicit model capabilities. o1 scores unreasonably poorly on a large portion of the preparedness tests, to the point where I would not take the tests seriously as assessments of what o1 is capable of doing, even at [...]