Learning features with two-layer neural networks, one step at a time

GT « Analyse, Algorithmique, Apprentissage »

Bruno Loureiro (ENS)

Title: 

Learning features with two-layer neural networks, one step at a time

Abstract: 

Feature learning – or the capacity of neural networks to adapt to the data during training – is often quoted as one of the fundamental reasons behind their unreasonable effectiveness. Yet, making mathematical sense of this seemingly clear intuition is still a largely open question. In this talk, I will discuss a simple setting where we can precisely characterise how features are learned by a two-layer neural network during the very first few steps of training, and how these features are essential for the network to efficiently generalise under limited availability of data.

Based on the following works: https://arxiv.org/abs/2305.18270, https://arxiv.org/abs/2402.04980, https://arxiv.org/abs/2410.18938