Using Extended Reality Simulations to Support Students in Understanding Abstract STEM Concepts

About This Grant

A solid foundation in science, technology, engineering, and mathematics (STEM) is crucial for graduates to thrive in the workplace. However, many foundational STEM concepts are abstract, counterintuitive, and difficult to understand. Therefore, STEM learners struggle to achieve the complete and deep understanding that would benefit them the most in their careers. This project aims to develop and investigate an extended reality (XR) learning environment that integrates interactive touch technology, augmented reality, and AI-driven engagement to support undergraduate students in learning abstract STEM concepts, particularly electromagnetism. It brings together advances in XR hardware, computer vision, and learning sciences to address persistent challenges in STEM education. While the learning benefits of the XR environment will be investigated in the context of electromagnetism, the XR environment promises applicability across a range of STEM fields. The project plans to develop and validate the XR environment through a series of intertwined technology and education research activities. Grounded in principles of embodied learning design, the project will develop technology-enhanced learning experiences consisting of the XR haptics and corresponding curricular materials. XR haptics will be provided by a versatile and cost-effective haptic device that delivers physically accurate force feedback. Furthermore, the XR headset worn by the learner will both augment and diminish their view of the real world. The augmentation will include an interactive visualization of the computer simulation illustrating the learning activity. Diminished reality algorithms will erase virtually elements of the real-world scene from the field of view of the learner to reduce visual clutter, with the aim of improving learning. XR environment user traces will be leveraged to monitor the learner’s level of engagement and to provide assistance when needed by playing back pre-recorded exemplary user traces. The XR environment will be developed with feedback from formative user studies that measure the accuracy of the force feedback, the quality of eye contact, the effectiveness of learner assistance, system usability, task load, sense of presence, cybersickness levels, and subjective user preference. The project plans to build three learning activities based on XR simulations of charged particles, bar magnets, and electromagnetic fields. The activities will be used in controlled studies that investigate whether the physically accurate haptic feedback is conducive to increased learning, based on pretests, posttests, and delayed posttests, for whom, based on the learner’s individual characteristics, and at what depth, based on the learners’ post-intervention argumentation. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.