Audio note: this article contains 182 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

I (Subhash) am a Masters student in the Tegmark AI Safety Lab at MIT. I am interested in recruiting for full time roles this Spring - please reach out if you're interested in working together!

TLDR

This blog post accompanies the paper "Language Models Use Trigonometry to Do Addition." Key findings:

We show that LLMs represent numbers on a helix
- This helix has circular features (sines and cosines) with periods <span>_T = [2,5,10,100]_</span> and a linear component
To solve <span>_a+b_</span>, we claim that LLMs manipulate the helix for <span>_a_</span> and <span>_b_</span> to create the helix for <span>_a+b_</span> using a form of the Clock algorithm introduced by Nanda, et al.
- This is conceptually akin to going from <span>_cos(a), cos(b)_</span> to <span>_cos(a+b)_</span>
- Intuitively, it's like adding angles on a [...]