Code and data can be found here

Executive Summary

We use data from Zhang et al. (2025) to measure LLM values. We find that our value metric can sometimes predict LLM behaviors on a test distribution in non-safety-relevant settings, but it is not super consistent.
In Zhang et al. (2025), "Stress testing model specs reveals character differences among language models," the authors generated a dataset of 43,960 chat questions. Each question gives LLMs a chance to express two different values, and an autograder scores how much the model expresses each value.
There are 3,302 values in the dataset, created based on what Claude expresses in the wild from Huang et al. (2025). Some examples are "dramatic craft," "structured evaluation," "self-preservation," and "copyright respect."
Using Zhang et al.'s data, we create value rankings for each LLM using a Bradley-Terry model. For example, Claude 3.5 Sonnet (new) values copyright respect and political sensitivity very highly, while Grok-4 does not.
We find that these value rankings are meaningfully predictive in some models: they can predict out-of-sample pairwise comparisons with accuracy ranging from 62.4% (o4-mini) to 88.1% (Grok-4), where chance is 50%.
We also use Petri to measure how models trade off between different values as [...]