Theory Talk - Sibylle Marcotte
June 5, 2026 10:30AM—11:30AM
Location:
In Person
-
Gates Hillman 8102
Speaker:
SIBYLLE MARCOTTE,
Postdoctoral Researcher
New York University
Understanding the geometric properties of gradient descent dynamics is a key ingredient in deciphering the recent success of very large machine learning models. A striking observation is that trained over-parameterized models retain some properties of the optimization initialization. This “implicit bias” is believed to be responsible for some favorable properties of the trained models and could explain their good generalization properties. In this talk, I will expose the definition and properties of “conservation laws” that define quantities that remain exactly preserved along gradient flows, regardless of the training data. Then I will explain how to find the exact number of independent conservation laws via Lie algebra computations. In the specific case of linear and ReLU networks, this procedure recovers the conservation laws known in the literature, and prove that there are no other laws. Finally, I will discuss how conservation laws can be used to rewrite high-dimensional training dynamics as intrinsic lower-dimensional dynamics. We establish that this reduction always holds for general ReLU networks via the path-lifting map, and that for linear networks it occurs precisely under relaxed-balanced initializations via the product map.
Joint work with Gabriel Peyré and Rémi Gribonval.
—
Sibylle Marcotte is a postdoctoral researcher at NYU working with Joan Bruna. She recently completed her PhD in the theory of deep learning at École Normale Supérieure in Paris, where she worked with Gabriel Peyré and Rémi Gribonval. Her research has been recognized with oral presentations at NeurIPS 2023 and ICML 2025, as well as a 2024 L’Oréal-UNESCO For Women in Science Young Talents France Award.
For More Information:
ochehab@andrew.cmu.edu