注意对视听语义整合加工的影响

摘要/Abstract

摘要： 采用2×3的被试内实验设计，将注意条件和目标刺激类型作为实验变量，考察了指向不同感觉通道的注意对视听语义整合加工的不同影响。结果发现，只有在同时注意视觉和听觉刺激时，被试对语义一致的视听刺激反应最快，即产生冗余信号效应。而在选择性注意一个感觉通道时，语义一致的视听刺激并不具有加工优势。进一步分析发现，在同时注意视觉和听觉时语义一致视听刺激的加工优势源自于其视觉和听觉成分产生了整合。也就是说，只有在同时注意视觉和听觉时，语义一致视听刺激才会产生整合，语义不一致视听刺激不会产生整合。而在选择性注意一个感觉通道时，不论语义是否一致，视听刺激均不会产生整合。

关键词: 注意, 多感觉整合, 语义联结, 竞争模型

Abstract: The present research used the square and the triangle as visual stimuli and spoken words as auditory stimuli to investigate the effect of attention on multisensory speech integration. The visual and auditory stimuli were semantically congruent or incongruent. The results from a discrimination task showed that: 1) In the condition of divided attention, participants required less time to react to audiovisual semantic-congruent targets than other types of targets; that is the redundant signal effect; 2) According to the race model, the redundant signal effect in the condition of divided attention originated from the multisensory integration. To conclude, the multisensory integration occurred when the attention was divided between visual and auditory modality only in the semantic-congruent audiovisual stimuli.

Key words: attention, multisensory integration, semantic congruence, race model

中图分类号:

B842.1

吕勇, 顾吉有. 注意对视听语义整合加工的影响[J]. 心理与行为研究, 2017, 15(5): 587-592.

LYU Yong, GU Jiyou. The Effect of Attention on Audiovisual Speech Integration[J]. Studies of Psychology and Behavior, 2017, 15(5): 587-592.

参考文献

刘强.(2010). 多感觉整合脑机制研究 (博士学位论文). 西南大学.
孙远路, 胡中华, 张瑞玲, 寻茫茫, 刘强, 张庆林. (2011). 多感觉整合测量范式中存在的影响因素探讨. 心理学报, 43(11), 1239-1246.
Frassinetti, F., Bolognini, N., & Làdavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Experimental Brain Research, 147, 332-343.
Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2), 127-138.
Haxby, J. V., Horwitz, B., Ungerleider, L. G., Maisog, J. M., Pietrini, P., & Grady, C. L. (1994). The functional organization of human extrastriate cortex: A PET-rCBF study of selective attention to faces and locations. Journal of Neuroscience, 14(11), 6336-6353.
Johnson, J. A., & Zatorre, R. J. (2005). Attention to simultaneous unrelated auditory and visual events: Behavioral and neural correlates. Cerebral Cortex, 15(10), 1609-1620.
Johnson, J. A., & Zatorre, R. J. (2006). Neural substrates for dividing and focusing attention between simultaneous auditory and visual events. NeuroImage, 31(4), 1673-1681.
Kawashima, R., O'Sullivan, B. T., & Roland, P. E. (1995). Positron-emission tomography studies of cross-modality inhibition in selective attentional tasks: Closing the “mind's eye”. Proceedings of the National Academy of Sciences of the United States of America, 92(13), 5969-5972.
Koelewijn, T., Bronkhorst, A., & Theeuwes, J. (2010). Attention and the multiple stages of multisensory integration: A review of audiovisual studies. Acta Psychologica, 134(3), 372-384.
Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y. F., Field, A. S., & Stein, B. E. (2002). Deactivation of sensory-specific cortex by cross-modal stimuli. Journal of Cognitive Neuroscience, 14(3), 420-429.
Laurienti, P. J., Kraft, R. A., Maldjian, J. A., Burdette, J. H., & Wallace, M. T. (2004). Semantic congruence is a critical factor in multisensory behavioral performance. Experimental Brain Research, 158(4), 405-414.
Lovelace, C. T., Stein, B. E., & Wallace, M. T. (2003). An irrelevant light enhances auditory detection in humans: A psychophysical analysis of multisensory integration in stimulus detection. Cognitive Brain Research, 17(2), 447-453.
Miller, J. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14(2), 247-279.
Miller, J. (1986). Timecourse of coactivation in bimodal divided attention. Perception & Psychophysics, 40(5), 331-343.
Mittag, M., Alho, K., Takegata, R., Makkonen, T., & Kujala, T. (2013). Audiovisual attention boosts letter-speech sound integration. Psychophysiology, 50(10), 1034-1044.
Raab, D. H. (1962). Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences, 24, 574-590.
Su, Y. H. (2014). Content congruency and its interplay with temporal synchrony modulate integration between rhythmic audiovisual streams. Frontiers in Integrative Neuroscience, 8, 92.
Talsma, D. (2015). Predictive coding and multisensory integration: An attentional account of the multisensory mind. Frontiers in Integrative Neuroscience, 9, 19.
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14(9), 400-410.
Talsma, D., & Woldorff, M. G. (2005). Selective attention and multisensory integration: Multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience, 17(7), 1098-1114.
Ten Oever, S., Sack, A. T., Wheat, K. L., Bien, N., & van Atteveldt, N. (2013). Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs. Frontiers in Psychology, 4, 331.
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45, 598-607.
Vatakis, A., & Spence, C. (2007). Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli. Perception & Psychophysics, 69(5), 744-756.

[1]	晏芹, 张清芳. 节奏在言语加工中的时间进程：来自元分析的证据[J]. 心理与行为研究, 2022, 20(5): 607-614.
[2]	李士一, 谢岩枫, 赵光, 白学军. 媒体多任务经验对不同注意模式下内隐记忆的影响[J]. 心理与行为研究, 2022, 20(4): 433-440.
[3]	刘艳艳, 李杰, 赵起超, 陈贝贝, 张禹. 三维动静态场景下返回抑制的扩散[J]. 心理与行为研究, 2022, 20(1): 22-28.
[4]	苏明宇, 王光明. STEAM训练提升小学生的注意抑制与注意集中能力[J]. 心理与行为研究, 2021, 19(5): 701-705.
[5]	乌仁其木格, 赵傲雪, 汪钰涵, 庄想灵, 马国杰. “找茬”任务中的视野偏好与参照策略[J]. 心理与行为研究, 2021, 19(2): 166-171.
[6]	高超, 李士一, 章鹏, 赵光. 基于双眼视差探讨深度知觉信息对背景线索效应的影响[J]. 心理与行为研究, 2021, 19(1): 16-21,29.
[7]	焦江丽, 刘毅, 闻素霞. 中英双语者情绪注意偏向中的双语优势效应[J]. 心理与行为研究, 2020, 18(5): 590-595.
[8]	车晓玮, 王凯旋, 上官梦麒, 李寿欣. 视觉工作记忆中注意模板的表征—来自EROS的证据[J]. 心理与行为研究, 2020, 18(3): 297-303.
[9]	吴慧芬, 黎晓娜, 张琼, 唐一纯, 刘华山. 因为注意，所以拥有—纯粹所有权效应内部机制探究[J]. 心理与行为研究, 2020, 18(2): 161-167,175.
[10]	李恒平, 何胜昔, 刘正奎. 快速阅读训练对基于视觉定向搜索的注意力影响：ERP研究[J]. 心理与行为研究, 2020, 18(1): 78-84.
[11]	于永菊. 焦虑和抑郁对正负性注意偏向的影响：注意控制的中介作用[J]. 心理与行为研究, 2020, 18(1): 121-127.
[12]	曾庆, 郑希付. 高低厌恶敏感性个体对不同情绪刺激的注意时程: 来自ERP研究的证据[J]. 心理与行为研究, 2019, 17(6): 743-749.
[13]	王庭照, 王凤丽, 敖勇前, 王丽. RSVP范式下情绪刺激对聋人与听力正常人注意瞬脱影响的比较研究[J]. 心理与行为研究, 2019, 17(6): 803-809.
[14]	刘丽, 李士一, 岳俊冰, 白学军. 位置-价值联结引发的注意捕获及其机制[J]. 心理与行为研究, 2019, 17(3): 289-295.
[15]	麦子峰, 孙瑞, 张群, 梁华, 马宁. 个体行为的昼夜节律效应：来自华南地区的证据[J]. 心理与行为研究, 2019, 17(2): 185-192.