Wednesday, May 31, 2017 - 1:30pm to 2:30pm
Location:Traffic21 Classroom 6501 Gates Hillman Centers
Speaker:GEORG SCHOENHERR, Ph.D. Student /GEORG%20SCHOENHERR
Neural networks are machine learning models represented by continuous programs of many parameters. Those parameters are generally optimized via gradient descent and related methods. However, those methods are indeed limited to tuning parameters of a given program. Choosing the program itself is an open problem. Programs are generally chosen by expert judgement, for computational convenience or by brute force search. I present "nonparametric neural networks", a framework for jointly optimizing program parameters and structure. Under this framework, we alternate between random local changes to the program and gradient-based parameter tuning and thus search over both components jointly. The search is enabled by defining a connected, continuous space over all program-parameter pairs, by penalizing programs according to their size, and by several optimization tricks. Presented in Partial Fulfillment of the CSD Speaking Skills Requirement.