Convergence Analysis and Efficient New Framework for Sampling-Based Optimal Control

About This Grant

Modern AI & robotics systems, such as general-purpose humanoid robots, must make intelligent decisions in real time, in unstructured environments. At the heart of these capabilities lies a fundamental problem: How can we compute optimal actions when such robotic systems are governed by highly complex dynamics (nonlinear, even discontinuous)? Traditional optimal control tools, while powerful, often fall short—they can be slow, unreliable, or rely on overly simplified assumptions about the real physics. This project aims to advance a promising category of methods known as Sampling-Based Optimal Control, which has recently gained popularity due to its flexibility (can easily handle complex systems), scalability (can leverage massively parallel computation on GPUs), and empirical success in solving complex robotic planning and control problems. However, these methods currently lack a solid theoretical foundation and systematic design principles. This project will fill that gap by developing new mathematical frameworks to understand, analyze, and improve these method—eventually leading to more reliable, efficient, and intelligent decision-making methods for real-world robotic and autonomous systems. By bridging theory and practice, this work supports NSF’s mission to promote transformative research in AI & robotics, and has the potential to impact a wide range of fields where autonomous systems must operate reliably and effectively in the real world. Optimal control for complex nonlinear and contact‑rich systems is notoriously nonconvex and often discontinuous, rendering classical optimization tools computationally burdensome and prone to suboptimal local minima. Sampling‑Based Optimal Control (SBOC) has emerged as an attractive alternative because its stochastic trajectory rollouts handle severe nonlinearity, discontinuities, and intricate cost landscapes while exploiting GPU‑based parallelism; it now underpins path planning, robotic control, and model‑based RL. Yet SBOC remains largely empirical: there is a lack of principled guidance on parameter tuning, sampling schedules, and, crucially, theoretical guarantees of convergence or sub‑optimality. This project closes these gaps through three coordinated thrusts. Thrust 1 develops asymptotic and finite‑time convergence theories that characterize when SBOC attains global optima and how performance scales with algorithmic and system parameters. Thrust 2 translates these insights into new, more efficient algorithms via automated hyper‑parameter optimization and a diffusion‑style annealing strategy that accelerates exploration while preserving convergence. Thrust 3 couples the resulting controllers with learned value functions, nominal policies, and dynamics models, yielding a powerful subroutine for model‑based reinforcement learning that unites the strengths of control and learning. The framework will be validated on demanding robotic benchmarks, including humanoid whole‑body locomotion and dexterous manipulation through extensive real hardware experiments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.