CAREER: Dynamical Analysis of Global Optimization Algorithms

About This Grant

The design of engineering systems – from chemical plants to molecular design – must be as effective as possible within limits imposed by costs, resources, physical laws, and safety rules. To find the best solution, engineers use computer programs called optimization solvers. Solvers rely on complex algorithms that need careful tuning by experts to perform well. This project will create a framework to better understand how these solvers work, focusing on how they rule out poor solutions and move toward the best one. By analyzing large amounts of data generated by the solvers and applying machine learning methods, the project will identify patterns to explain solver behavior. The results will help engineers interpret solver decisions and enable solvers to automatically improve their own performance, becoming faster and more accurate without manual adjustment. This data-driven approach can strengthen industrial automation systems, support materials and drug design, and speed up the development of new processes and products. In addition, the project will support education by building data science and computational skills in chemical engineering courses and by encouraging younger students to explore careers in STEM. This project adopts a novel dynamical-systems perspective to represent, learn, and optimize global optimization algorithms. In this framework, optimization algorithms are modeled as dynamical systems whose states evolve in function spaces defined over the feasible domain, with their evolution governed by linear operators acting on these spaces. This representation enables a unified and principled analysis of diverse global optimization methods. Specifically, the dynamics of widely used global optimization algorithms in process systems engineering and machine learning—such as stochastic gradient Langevin dynamics, branch-and-bound, outer approximation, and black-box Bayesian optimization—will be learned directly from solver snapshot data or historical execution traces. By identifying the underlying linear operators that characterize algorithm evolution, the associated dynamic modes - functions that evolve linearly under the learned dynamics - can be extracted. These modes provide mechanistic insights into solver behavior, including pruning, fathoming, and optimality gap reduction, while also enabling accelerated convergence through informed extrapolation of solver trajectories. Algorithm tuning parameters are interpreted as control inputs that influence computational performance metrics such as accuracy and efficiency. Automatic solver tuning is then formulated as an optimal control problem defined over the learned algorithm dynamics, allowing systematic and data-driven optimization of solver performance without manual heuristics. The resulting framework is fully end-to-end, encompassing data collection, operator learning, mode extraction, and solver tuning, and is broadly applicable across classes of global optimization algorithms. To demonstrate its practical advantages and scientific impact, the enhanced solvers will be applied to molecular design tasks - benchmarking against existing generative modeling approaches - as well as transition pathway optimization in molecular simulations to uncover chemical reaction mechanisms. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.