OAC Core: Revolutionizing Data Movement on DPU-Powered HPC and AI Cyberinfrastructures

About This Grant

Efficient data movement is a critical challenge in high-performance computing (HPC) and artificial intelligence (AI) cyberinfrastructures due to the massive volumes of data generated by modern data-intensive applications. Existing methods often struggle with performance bottlenecks, particularly when transferring data across parallel and distributed computing environments. To address these limitations, this project -- the Open DPU-Offloading data Transfer Architecture (OpenDOTA) -- provides a framework that leverages Data Processing Units (DPUs) to accelerate data movement. By enhancing efficiency in DPU-powered systems, OpenDOTA aims to advance scientific simulations, drive AI advancements, and strengthen computational research infrastructure. The project fosters collaboration and contributes to the evolution of state-of-the-art data movement technologies, benefiting a wide range of users in academia and industry. This project focuses on designing OpenDOTA as a high-performance, scalable framework for DPU-offloaded data movement in HPC and AI cyberinfrastructures. The research is structured around three key thrusts: (1) Adaptive point-to-point data movement, which employs diverse offloading strategies to optimize data transfer over DPUs; (2) Accelerating collective communication by leveraging advanced DPU offloading techniques to improve scalability; and (3) Deep reinforcement learning (DRL)-based optimization, which dynamically adapts data movement strategies for maximum performance. By integrating these approaches, OpenDOTA offers a comprehensive solution to existing data movement challenges, paving the way for scalable, high-performance applications across HPC and AI domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.