Focusing and Switching Attention During a Mixed Modality Auditory-Visual Vigilance Task

There are many scenarios where it is advantageous to prioritize and selectively attend to a single sensory source as well as flexibly engage, disengage, and re-engage with multiple sensory modalities, such as when operating a vehicle. The influences of simultaneous stimuli from different sources and from multiple sensory modalities (e.g., visual and auditory) have been shown to divide attention, which degrades performance. In addition, time-on-task cognitive fatigue can compromise performance and the ability to ignore task-irrelevant information. The aims of this study were to examine whether vigilance-related declines in performance are attributable to supramodal resources that are shared across multiple sensory modalities and to characterize the main source of false alarm errors over time. Stimuli were presented visually and audibly, and participants were instructed to respond to either the visual targets only or the auditory targets only (i.e., focusing on a single modality) or both visual and auditory targets (i.e., alternating attention between modalities). The auditory and visual stimuli involved pairing letters and sounds (phoneme /b/ with the grapheme b and phoneme /p/ with the grapheme p), resulting in non-targets that could be either modality-relevant non-targets (i.e., the same modality as the target and not the target signal), modality-irrelevant non-targets (i.e., a different modality of the target and not paired with the critical target), or modality-irrelevant target-congruent (i.e., a different modality, but sharing a letter-sound correspondence with the critical signal). While significant effects of cognitive fatigue over time were not detected, there was reduced accuracy when switching attention between auditory and visual targets compared to focusing attention on a target from a single modality. Also, when focusing attention on a single modality, false alarms to modality-relevant non-target items were substantially higher than non-targets