Reading notes on lazy and rich regimes
2026-08-21
I have not published much yet. These are papers I keep on the desk, mostly around the lazy–rich transition. They feed the working notes on network richness.
- Lénaïc Chizat, Edouard Oyallon, and Francis Bach. On Lazy Training in Differentiable Programming. NeurIPS, 2019. arXiv:1812.07956.
- Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural Tangent Kernel: Convergence and Generalization in Neural Networks. NeurIPS, 2018. arXiv:1806.07572.
- Dhruva Karkada. The lazy (NTK) and rich (μP) regimes: a gentle tutorial. 2024. arXiv:2404.19719.
- Greg Yang and Edward J. Hu. Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks. ICML, 2021. arXiv:2011.14522.
- Mario Geiger, Stefano Spigler, Arthur Jacot, and Matthieu Wyart. Disentangling feature and lazy training in deep neural networks. 2020. arXiv:1906.08034.