bet365 online sports betting app title: Theoretical basis of deep learning : Optimization, generalization and implicit bias of gradient algorithm
Reporter:Li Jian Tsinghua bet365 online sports betting app,Professor, PhD supervisor
Reporting location:Computer Building313
bet365 online sports betting app time：2024year9month25Sunday (Wednesday), PM4Point
bet365 online sports betting app Introduction:Deep learning has achieved great success in applications，However, its related theoretical basic bet365 online sports betting app is relatively lagging behind。The training of deep neural networks is a highly non-convex optimization problem，But the simple stochastic gradient method can find a way to not only minimize the training error but also show strong generalization ability to unseen data。This kind of generalization ability cannot be explained by classic machine learning theory。Recently，Researchers find that gradient methods may not converge to a stable point，And lose the sharpness of the landscape (Hessian) may fluctuate and enter a state known as the edge of stability。These behaviors are inconsistent with several assumptions widely adopted in the field of classical optimization。What bias does the gradient basis algorithm introduce in neural network training？What is the relationship between this bias and the existence of adversarial samples？These are questions that cannot be answered by classic optimization theory and statistical learning theory。These new problems require new theoretical explanations and mathematical foundations。In this bet365 online sports betting app，From the perspective of gradient optimization method，Optimizing the behavior of trajectories through interpretation and analysis，Let’s get into these basic theoretical issues in deep learning，And analyze and explain the above issues from the perspective of gradient method。
About the speaker:Li Jian，Distinguished Professor of the Institute of Cross-Information, Tsinghua bet365 online sports betting app，Doctoral Supervisor。Research direction is theoretical computer science、Basic Theory of Artificial Intelligence、Database、Fintech。Published in mainstream international conferences and magazines100More papers, and won the top database conferenceVLDBHe European Algorithm Annual ConferenceESA's Best Paper Award, Database Theory ConferenceICDTBest Newcomer Award、Multiple papers were selected for oral presentations or highlight papers。Selected into the National Youth Talent Program。Has hosted and participated in many Natural Science Foundation projects，And multiple corporate cooperation projects including Baidu、Ant Financial、Today’s headlines、Yifangda、Huatai Securities, etc.。
