The Pontryagin maximum principle and the training of Runge-Kutta neural networks

In residual neural networks and related NN architectures, supervised learning problems can be reformulated as optimal control problems governed by discrete-in-time nonlinear evolution models. This talk is devoted to the analysis and solution of these problems in the framework of a discrete version of the Pontryagin maximum principle and of neural networks with Runge-Kutta (RK) structure. In particular, a sequential quadratic Hamiltonian (SQH) method for solving the corresponding supervised learning problems is presented. Convergence properties of the SQH scheme are investigated theoretically and numerically, and results of numerical experiments are presented that demonstrate the advantageous performance of the SQH learning algorithm. Contatto: marco.verani@polimi.it