This is a research update for an on-going replication of no-CoT evals done as part of the Second Look Fellowship. In following posts, we will run more comprehensive replications of previous work and release open source tooling for no-CoT eval elicitation. Code can be found here.

tl;dr

We replicate experiments from Greenblatt 2025 and Greenblatt 2026 on GPT-6-Astra, on the same items and protocol as our previous update on Fable 5, Opus 5, Opus 4.5, and GPT-5.6-Sol, plus Gemini 3.1 Pro, Kimi k3, and Fable 5.1.
We find that Astra is a qualitative jump in no-CoT capabilities over all datasets.
- 4-hop questions: Astra achieves 31% at baseline, where every other model tested scores at 1-3%
- 3-hop questions: 70% against previous best of 22% (Gemini 3.1 Pro)
- Neel Nanda and Rohan Subramani report the same jump independently. Our work qualitatively replicates these results.
Astra sees more uplift from filler tokens and repeats than previous models
- 4-hop performance is doubled from baseline (31%) to peak filler condition (63% at )
- 3-hop accuracy jumps from 70% to 85%
  - Filler tokens and problem repeats raise accuracy monotonically across the full range we tested
- Dylan Xu, SebastianP, & Alek [...]