Please share this with anyone doing AI research with 3rd party providers so that they can ensure their research won’t be corrupted.

When you ask OpenRouter[1] to give you tokens from a given model, OpenRouter sends your request to a random available provider.
OpenRouter providers have variable quality. Ensuring that your provider is high quality is really difficult.
There is precedent for an AI safety paper accepted to NeurIPS having its core results entirely overturned by these issues.
A review of influential AI Safety research codebases that use OpenRouter for their reported results found that 31/32 (97%) of them use OpenRouter unsafely.[2]
Researchers who wish to do research using OpenRouter or similar providers should take precautions to minimize the risks to their research,[3] though the current selection of providers is insufficient for ideal scientific reliability.
Replicators should test whether results hold up when these bugs are fixed.
Core AI Safety codebases should make fixes so that downstream users are able to implement best practices.

This is a tangent I took for a few days during the Pivotal AI Safety Research Fellowship. I’m doing an AI Control project mentored by Adam Kaufman and James Lucassen of Redwood Research and partnered with Aniruddh [...]