AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart

Physics provides a rigorous framework for understanding machine learning by modeling neural networks as complex systems with rough energy landscapes. Deep architectures overcome the "poverty of stimulus" by utilizing implicit biases to discover hierarchical abstractions, effectively learning generative rules from limited data. While standard models predict in token space, introspective algorithms that predict in latent space achieve superior sample efficiency by reducing noise and focusing on high-level conceptual relationships. These models demonstrate that as training data increases, networks progressively capture more abstract, long-range correlations, explaining observed scaling laws. Matthieu Wyart, a professor at Johns Hopkins and EPFL, applies this statistical physics perspective to argue that intelligence emerges from the ability to reduce dimensionality through hierarchical coarse-graining, offering a path toward more efficient, creative, and human-like machine learning systems.