AI & Mathematics
Day 2 (Tuesday 23 April) @ 13:30–15:00
Gaurav Rattan (University of Twente)
Graph-Theoretic Principles for Graph Learning Models
Graph Neural Networks (GNNs) are deep learning models for machine learning tasks in the domain of graphs and networks. Despite the widespread popularity of GNNs in the practical setting, the mathematical principles underlying the design and analysis of these models are hardly well-understood. This naturally calls for a first-principles approach to machine learning on graphs, rooted in classical graph theory, instead of adapting existing learning models for other data domains such as vectors, text, and images.
In this talk, we describe our work towards establishing the mathematical foundations for GNNs in terms of the existing graph structure theory and algorithmic graph theory, in particular, the connection to the fundamental Graph Isomorphism problem. We show that a standard message-passing GNN is essentially a neuralized version of the Weisfeiler-Leman algorithm, a classic graph isomorphism testing algorithm. We highlight the key role played by mathematical logic in expanding this connection into a powerful framework for GNN design. Finally, we link together the beautiful theory of graph homomorphism numbers, enunciated by Lovasz and others, with the aforementioned Weisfeiler-Leman algorithm and GNNs: This forms a starting point for analyzing the generalization properties of GNNs.

Jemima Tabeart (Eindhoven University of Technology)
Machine learning challenges and opportunities in variational data assimilation
Data assimilation methods combine information from measurements and dynamical systems in order to obtain an improved state estimate of a system of interest. This is often used to find suitable initial conditions for forecasting e.g. in numerical weather prediction. Such methods can be very powerful, and are becoming more popular outside the geosciences. However, developing the necessary dynamic and error models can be difficult, particularly for novel and more data-driven applications. In this talk I will introduce variational data assimilation methods, and discuss some aspects of the problem which are challenging for standard mathematical and physics-based approaches. I will highlight some problems where machine learning is already being used successfully within data assimilation for weather prediction, and present some more general open problems for other applications where machine learning could provide a missing puzzle piece.

Maximilian Engel (University of Amsterdam)
Characterizing Dynamic Stability of Stochastic Gradient Descent
For overparameterized optimization tasks, such as the ones found in modern machine learning, global minima are generally not unique. In order to understand generalization in these settings, it is vital to study to which minimum an optimization algorithm converges. The possibility of having minima that are unstable under the dynamics imposed by the optimization algorithm limits the potential minima that the algorithm can find.
In our work, we characterize the global minima that are dynamically stable/unstable for both deterministic and stochastic gradient descent (SGD), introducing a characteristic Lyapunov exponent around a global minimum and rigorously proving that the sign of this Lyapunov exponent determines whether SGD can accumulate at the respective minimum.
This is joint work with Dennis Chemnitz (FU Berlin).
