How Are Audiovisual Simultaneity Judgments Affected by Multisensory Complexity and Speech Specificity?

While stimulus complexity is known to affect the width of the temporal integration window (TIW), a quantitative evaluation of ecologically highly valid stimuli has not been conducted. We assumed that the degree of complexity is determined by the obviousness of the correspondence between the auditory onset and visual movement, and we evaluated the audiovisual complexity using video clips of a piano, a shakuhachi flute and human speech. In Experiment 1, a simultaneity judgment task was conducted using these three types of stimuli. The results showed that the width of TIW was wider for speech, compared with the shakuhachi and piano. Regression analysis revealed that the width of the TIW depended on the degree of complexity. In the second experiment, we investigated whether or not speech-specific factors affected the temporal integration. We used stimuli that either contained natural-speech sounds or white noise. The results revealed that the width of the TIW was wider for natural sentences, compared with white noise. Taken together, the width of the TIW might be affected by both the complexity and speech specificity.