When working through a problem, OpenAI's o1 model will write a chain-of-thought (CoT) in English. This CoT reasoning is human-interpretable by default, and I think that this is hugely valuable. Assuming we can ensure that these thoughts are faithful to the model's true reasoning, they could be very useful for scalable oversight and monitoring. I'm very excited about research to help guarantee chain-of-thought faithfulness.[1]

However, there's a impending paradigm for LLM reasoning that could make the whole problem of CoT faithfulness obsolete (and not in a good way). Here's the underlying idea, speaking from the perspective of a hypothetical capabilities researcher:

Surely human-interpretable text isn't the most efficient way to express thoughts. For every token that makes some progress towards the answer, you have to write a bunch of glue tokens like "the" and "is"—what a waste of time and compute! Many useful thoughts may even be inexpressible in [...]