Multisensory integration of speech sounds with letters vs. visual speech: only visual speech induces the mismatch negativity.

ABSTRACT: Numerous studies have demonstrated that the vision of lip movements can alter the perception of auditory speech syllables (McGurk effect). While there is ample evidence for integration of text and auditory speech, there are only a few studies on the orthographic equivalent of the McGurk effect. Here, we examined whether written text, like visual speech, can induce an illusory change in the perception of speech sounds on both the behavioural and neural levels. In a sound categorization task, we found that both text and visual speech changed the identity of speech sounds from an /aba/-/ada/ continuum, but the size of this audiovisual effect was considerably smaller for text than visual speech. To examine at which level in the information processing hierarchy these multisensory interactions o

SUBMITTER: Stekelenburg JJ 

PROVIDER: S-EPMC5969231 | biostudies-literature | 2018 May

REPOSITORIES: biostudies-literature