Valentin Leplat

Valentin Leplat
Valentin Leplat
Skoltech Senior Research Scientist
Recent successes of higher-order methods in machine learning
The recent progress in computer technologies and telecommunications allows collecting very large volumes of information. The data continuously grows year after year and processing such amounts of data becomes a key challenge. In the data sciences community, we are usually interested in identifying the underlying structure of the data and extracting meaningful information; in the last decade the most successful approaches for achieving this goal are based on deep learning (DL) techniques which commonly involves solving large scale optimization problems. The standard methods for solving such problems are based on first-order schemes such as the famous stochastic gradient descent (SGD), and this for many convincing reasons: they are computationally cheap, usually simple to implement and demonstrated a lot of practical successes for the training of numerous DL models. On their side, higher-order methods, including second-order methods, are more rarely considered in DL, although benefiting from many strengths such as faster convergence (per iteration) and frequent explicit regularization step-size. Moreover, many scientific fields have successfully used second-order methods.
This talk briefly presents some recent and successful developments and use of higher-order methods in deep learning and machine learning.

Valentin Leplat received engineering degrees in mechanical engineering from Gramme Institute, Liège, Belgium, in 2012 and in computer sciences and applied mathematics engineering from University of Mons, Belgium in 2017. He worked for six years as an aerospace engineer at SONACA, Gosselies, Belgium. He completed his Ph. D. in applied mathematics in January 2021 on the topic of Nonnegative Matrix Factorizations (NMF), associated with the Department of Mathematics and operations research at University of Mons, Belgium. In 2021, he was a postdoctoral research associate in applied mathematics at Université Catholique de Louvain (Belgium). Since October 2021, he is working at Skoltech (Moscow) as a senior research scientist on the topics of machine learning, stochastic optimization and tensors decompositions.