Effective Computational Methods for Training of Neural Networks via Exploration-Exploitation-Determination Framework

About This Grant

This project addresses one of the fundamental challenges in artificial intelligence (AI): training of neural networks (NNs). NNs are a core component of modern AI models and are widely used in numerous applications across science, engineering, and industry. Training refers to a learning process of AI models. It is notoriously challenging due to the nonlinear and nonconvex nature of NNs. The state-of-the-art methods frequently fail to produce satisfactory results and exhibit unstable behaviors, limiting the accuracy and reliability of AI models. This project will develop effective training methods that significantly improve the training performance over the state-of-the-art, overcoming the current limitations. The broader impacts include creating educational opportunities for undergraduates through a summer research program, developing professional training for K-12 educators on computational mathematics for AI, and establishing public engagement through interactive demonstrations and online resources, all of which will broaden participation in computing and improve public understanding of the role of mathematics in AI. The project develops a novel exploration-exploitation-determination (EED) framework for training neural networks that uniquely combines both local and nonlocal information. Four objectives are: (1) establishing the EED framework for two-layer neural networks by utilizing mathematical analysis of gradient flow dynamics and nonlocal effects; (2) developing a layer-wise training strategy for deep neural networks that sequentially trains each hidden layer; (3) extending the framework to handle noisy and corrupted data through l1-norm minimization; and (4) validating the methods through applications to scientific machine learning tasks including operator learning for PDEs and flow map learning for data-driven discovery of dynamical systems. The computational methods will be rigorously analyzed mathematically, and the resulting algorithms will be made publicly available to enhance reproducibility and maximize impact across multiple scientific disciplines. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.