We're posting lots of papers written with GPT-6 Astra on generalization properties of bayesian neural net learning. The following picture emerges:

For various reasonable priors on MLP weights, the induced prior on functions is a circuit prior: for fixed depth, the ordering of functions by negative log prior is roughly their ordering by minimal circuit width. From a quantitative version of this statement, one gets the following sample complexity bound: if the function is implemented by a circuit of width , then conditioning on train data points is sufficient to get test accuracy.
- These results hold as long as is realizable; in particular, for arbitrarily overparametrized NN-bayes.
- Probably the main caveat is that these bounds have exponential dependence in the depth of the circuit implementing . One can get only polynomial dependence in depth by restricting the prior to only neural nets satisfying a robustness assumption.
For various other reasonable priors on MLP weights,, one gets bad sample complexity when very overparametrized: when learning some functions , even after seeing train data points, the learner is still likely to have at least some constant test error rate.

GPT-6 Astra [...]

The original text contained 8 footnotes which were omitted from this narration.

---

First published:
October 21st, 2026

Source:
https://www.lesswrong.com/posts/oePzacZn6wNcJPvH3/circuit-priors-in-nn-bayes-more-results

---

“Circuit priors in NN-bayes: more results” by Kaarel, Dmitry Vaintrob

LessWrong (30+ Karma)