The Goldilocks Zone and Geometric Features of High-Dimensional Parameter Spaces

Content Provider	Semantic Scholar
Author	Chang, Jeffrey T.
Copyright Year	2019
Abstract	Recently, it has been proposed that neural networks exhibit a ‘Goldilocks Zone’: a region in parameter space of low loss and high generalization accuracy, located in a hyper-annulus where the sum of squared weights is not too small and not too large. In this project, I report that neural networks trained on the CIFAR-10 dataset exhibit this phenomenon. I also explain some of the observed behaviors using geometric arguments about high-dimensional spaces.
File Format	PDF HTM / HTML
Alternate Webpage(s)	http://cs229.stanford.edu/proj2019aut/data/manual/chang.pdf
Language	English
Access Restriction	Open
Content Type	Text
Resource Type	Article