Audio-visual effects across different sources of attentional control

About This Grant

The human brain is constantly confronted with sensory inputs from multiple modalities. To form a coherent perceptual experience, these distinct sensory signals need to be integrated even though they are received by distinct sensory systems. One possibility is that these signals are combined on the basis of spatial location. In most of our natural experience, the source of sensory signals that are related come from the same location, making it an ideal basis on which to integrate information to form a coherent perception of the world around us. The work proposed here aims to test whether spatial location serves this function for visual and auditory attention. The proposed work tests this mechanism in several tasks and for attention driven by the stimuli, internal control, and prior experience. The project also includes the development of an EEG methods boot camp and other outreach efforts that may aid in workforce development and increase public awareness of the research. In detail, prior research demonstrating spatial selectively for both visual and auditory attention has often been taken as evidence for a ‘supra-modal’ spatial attention system that is tuned to operate similarly across the different senses. The current proposal aims to test key tenets of this spatial attention theory focusing on audiovisual interactions using a combination of psychophysics and EEG. First, it aims to examine the time course of spatial exogenous attention and assesses whether cross-modal attention follows similar temporal dynamics as unimodal attention. Second, it aims to investigate the interactions between exogenous cross-modal attention and endogenous, sustained visual-spatial selection. Finally, the proposed works aims to quantify how selection history in one modality (e.g., audition) affects the spatial prioritization in another modality (e.g., vision) using incidental learning paradigms. The results of the proposed studies have the potential to bring forward a new understanding of how multimodal information is processed and integrated in the brain to support effective stimulus processing, advancing current models of attention and multisensory integration. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.