Frequency principle in understanding the generalization of deep learning and the depth effect

Zhi-Qin John Xu, Shanghai Jiao Tong University
Event time: 
Tuesday, October 20, 2020 - 9:00am
Location: See map
Event description: 

Abtract:  We demonstrate a very universal frequency principle that deep neural networks learn low frequency faster on high-dimensional benchmark datasets such as MNIST/CIFAR10 and deep neural networks such as VGG16. Then, we utilize the frequency principle to empirically provide a promising mechanism to understand why deeper learning is faster, that is, we propose a deep frequency principle that the effective target function for a deeper hidden layer biases towards lower frequency during the training.

Event Type: 
Applied Mathematics