FW-HTF-R/Collaborative Research: Human-Robot Sensory Transfer for Worker Productivity, Training, and Quality of Life in Remote Undersea Inspection and Construction Tasks

About This Grant

This Future of Work at the Human-Technology Frontier: Core Research project will create a novel interface for remote operation of undersea robots and customize it to the needs of offshore industries and workers. The novel interface integrates robot sensor readings and high-speed predictive simulations of hydrodynamic forces to create an immersive mixed reality (MR) display. In addition to augmented video images of the robot's surroundings, the interface converts measurements of water flow rates, hydrostatic pressure, ambient temperature, and other variables into tactile sensations for the operator. Likewise, the interface will render natural movements of the operator's body into control commands to the robot. The goal is human-robot "sensory transfer," that is, seamless translation of perceptions and actions between the operator and the robot. The goal of this project is to develop and match the capabilities of the interface to industry and worker needs. One anticipated benefit is to reduce the extensive training currently required for operators, thereby increasing access to these jobs while reducing industry training expenses and downtime due to personnel shortages. The project will study the most effective way to improve worker performance, safety, and quality of life, and by requiring a diverse set of subjects, will show how such human-robot interfaces can expand economic opportunity to broad sections of society. The interface can also be used in a purely virtual mode as a training tool. The project will examine the use of this capability to recruit workers from adjacent fields, such as construction. Offshore applications that would directly benefit from this project include subsea infrastructure inspection, geological surveys, marine habitat monitoring, pollution assessments, ship-hull inspections, unexploded ordnance surveys, contraband detection, aquaculture monitoring, search and rescue, and archaeological exploration and surveys. An increase in extreme weather and rising sea levels will place increasing demands on offshore operations to protect and repair coastal damage. Similarly offshore sustainable energy infrastructure such as wind, wave, or tidal generators will increase the demand for undersea inspection, construction, and maintenance. This project will reconceptualize future subsea industry by advancing knowledge of underwater Human-Robot Interaction (HRI) in under-explored subsea workplaces, illuminating socioeconomic features and adult-learning needs of workforce transformation to subsea industry, and establishing academia-industry-government partnerships for improving performance, safety, and societal outcomes of subsea works. Novel human-robot sensory transfer methods are suggested for reliability against conditions unique to subsea. These methods will support fast and accurate reconstruction of subsea workplaces. MR will be used to generate human-perceivable simulation of remote subsea workplaces in real time based on feedback from a novel robotic sensing and data transmission system. Motion capture will be created for easier navigation of remotely operated vehicles (ROVs). This research will establish new knowledge on motivational and educational determinants of introducing easy-to-use collaborative ROVs as part of a transformative workforce for future subsea robot operations, through extensive participation from industrial partners. The assessment will integrate techniques from psychometric and behavioral sciences as well as engineering and human factors. The work will also pioneer the development of a future subsea job framework for integration of ROVs into a participatory delivery of core subsea services. The economic benefits of robotic adoption will be estimated based on demand projection and elasticity estimation. This research will transform the frontiers of human-technology partnership in the context of the future subsea industry, reposition workforce threatened by automation in other domains, enhance future workers’ safety and well-being, and improve subsea operation performance, thus enhancing the long-term sustainable ocean exploration. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.