机器学习实验室博士生系列论坛（第二十一期）——Understanding the shallow neural networks under the teacher-student setting.
报告人：Kun Chen (PKU)
时间：2021-12-22 15:10-16:10
地点：北大理科一号楼1513会议室 & 腾讯会议 761 4699 1810
Abstract: Deep learning has made substantial progress in many applications, including computer vision, nature language processing and speech recognition. The mysterious generalization ability of the neural networks, which we concern most, usually connects with the optimization closely.
In this talk, we focus on the shallow neural networks with the teacher-student setting, where the generalization can be treated as a part of optimization. We will mainly discuss the landscape properties of the neural networks under several different situations, as well as the training dynamics. Specifically, we will first give a brief introduction about the background and main results of the teacher-student setting with various of data distribution and number of nodes. Then we further talk about the recovery of parameters and the spurious local minima of the loss function under certain ReLU activation function.
