The Influence of Language Differences on the Cross-Modal Priming Effect of Emotion

Abstract

Abstract: The cross-modal priming paradigm was used to investigate the interactive effects of speech and facial emotion processing as well as the influences of linguistic differences (native language: Chinese; non-native language: English). In experiment 1, the emotional verbal word was served as the prime stimulus and the emotional face was served as the target stimulus. The results showed that participants achieved better performances on judging facial emotion under the condition of Chinese prime stimulus than that under the condition of English prime stimulus. Besides, under the circumstance of positive stimulus priming, the processing of verbal emotion could influence that of facial emotion. As for experiment 2, the emotional face was adopted as the prime stimulus while the emotional verbal word was used for the target stimulus. It was found that under Chinese target stimulus, participants had better performances on judging the verbal emotion than that under the English target stimulus. Moreover, the processing of facial emotion was able to activate that of verbal emotion in condition of positive stimulus priming. The results indicate that verbal emotion and facial emotion processing interacts with each other; however, such interaction is only reliable with positive primes. In addition, the emotional function of the native language is different from that of the non-native language.

Key words: emotional words, emotional face, native language, non-native language, cross-modal prime paradigm

摘要： 采用跨通道启动范式，探讨言语与面孔情绪加工的相互影响效应，以及语言差异（母语：汉语；非母语：英语）在其中的影响作用。实验1以言语情绪词为启动刺激，面孔情绪为目标刺激，结果发现，相比于英语启动刺激条件，在汉语启动刺激条件下的面孔情绪判断具有更好的表现；在积极情绪启动条件下，言语情绪刺激能够启动面孔情绪刺激。实验2以面孔情绪为启动刺激，言语情绪词为目标刺激，结果发现，相比于英语目标刺激条件，在汉语目标刺激条件下的言语情绪判断具有更好的表现；在积极情绪启动条件下，面孔情绪刺激能够启动言语情绪刺激。研究结果表明，言语情绪与面孔情绪的加工能够相互影响，但这种相互关系仅表现在积极情绪启动条件下。此外，母语和非母语在情绪功能上具有差异性。

关键词: 言语情绪词, 面孔情绪, 母语, 非母语, 跨通道启动范式

CLC Number:

B842

ZHANG Fangwen, MA Xiao, MA Xie. The Influence of Language Differences on the Cross-Modal Priming Effect of Emotion[J]. Studies of Psychology and Behavior, 2020, 18(4): 452-459.

张芳文, 马骁, 马谐. 语言差异对跨通道情绪启动效应的影响[J]. 心理与行为研究, 2020, 18(4): 452-459.

References

白露, 马慧, 黄宇霞, 罗跃嘉. (2005). 中国情绪图片系统的编制——在46名中国大学生中的试用. 中国心理卫生杂志, 19(11), 719-722, doi: 10.3321/j.issn:1000-6729.2005.11.001
常翼, 许晶, 史娜. (2010). 卡通表情自动加工的失匹配负波研究. 中华行为医学与脑科学杂志, 19(6), 526-528, doi: 10.3760/cma.j.issn.1674-6554.2010.06.017
陈俊, 李佳南. (2016). 汉-英双语者的情感启动: 二语熟练程度对非对称性的影响. 外语教学与研究, 48(3), 396-408
蒋重清, 杨丽珠, 刘颖. (2007). 阈下情绪STROOP效应发展特点. 心理学报, 39(2), 242-248
姜帆, 刘永兵. (2015). 听觉启动刺激具体性对外语词语音位表征的影响. 现代外语, 38(4), 526-533
李芳, 张海燕, 田锐, 丁亮. (2015). 前瞻记忆的情绪一致性反转效应. 心理与行为研究, 13(2), 198-204, doi: 10.3969/j.issn.1672-0628.2015.02.008
刘文娟, 沈曼琼, 李莹, 王瑞明. (2016). 情绪概念加工与情绪面孔知觉的相互影响. 心理学报, 48(2), 163-173
吕勇, 张伟娜, 沈德立. (2010). 不同愉悦度面孔阈下情绪启动效应: 来自ERP的证据. 心理学报, 42(9), 929-938
王若茵, 范宁. (2016). 任务与字频对汉字情绪启动反转效应的影响. 心理科学, 39(3), 559-565
汪向东, 王希林, 马弘. (1999). 心理卫生评定量表手册 (增订版). 北京: 中国心理卫生杂志社.
王一牛, 周立明, 罗跃嘉. (2008). 汉语情感词系统的初步编制及评定. 中国心理卫生杂志, 22(8), 608-612, doi: 10.3321/j.issn:1000-6729.2008.08.014
Anooshian, L. J., & Hertel, P. T. (1994). Emotionality in free recall: Language specificity in bilingual memory. Cognition and Emotion, 8(6), 503-514, doi: 10.1080/02699939408408956.
Bach, D. R., Schmidt-Daffy, M., & Dolan, R. J. (2014). Facial expression influences face identity recognition during the attentional blink. Emotion, 14(6), 1007-1013, doi: 10.1037/a0037945.
Campanella, S., & Belin, P. (2007). Integrating face and voice in person perception. Trends in Cognitive Sciences, 11(12), 535-543, doi: 10.1016/j.tics.2007.10.001.
Carroll, N. C., & Young, A. W. (2005). Priming of emotion recognition. The Quarterly Journal of Experimental Psychology Section A, 58(7), 1173-1197, doi: 10.1080/02724980443000539.
Chan, E., Ybarra, O., & Schwarz, N. (2006). Reversing the affective congruency effect: The role of target word frequency of occurrence. Journal of Experimental Social Psychology, 42(3), 365-372, doi: 10.1016/j.jesp.2005.04.008.
Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., & Lepore, F. (2008). Audio-visual integration of emotion expression. Brain Research, 1242, 126-135, doi: 10.1016/j.brainres.2008.04.023.
De Gelder, B., Böcker, K. B. E., Tuomainen, J., Hensen, M., & Vroomen, J. (1999). The combined perception of emotion from voice and face: Early interaction revealed by human electric brain responses. Neuroscience Letters, 260(2), 133-136, doi: 10.1016/S0304-3940(98)00963-X.
Dewaele, J. M. (2004). The emotional force of swearwords and taboo words in the speech of multilinguals. Journal of Multilingual and Multicultural Development, 25(2-3), 204-222.
Dewaele, J. M., & Pavlenko, A. (2002). Emotion vocabulary in interlanguage. Language Learning, 52(2), 263-322, doi: 10.1111/0023-8333.00185.
Egner, T., & Hirsch, J. (2005). Cognitive control mechanisms resolve conflict through cortical amplification of task-relevant information. Nature Neuroscience, 8(12), 1784-1790, doi: 10.1038/nn1594.
Eilola, T. M., Havelka, J., & Sharma, D. (2007). Emotional activation in the first and second language. Cognition and Emotion, 21(5), 1064-1076, doi: 10.1080/02699930601054109.
Ethofer, T., Anders, S., Erb, M., Droll, C., Royen, L., Saur, R., … Wildgruber, D. (2006). Impact of voice on emotional judgment of faces: An event-related fMRI study. Human Brain Mapping, 27(9), 707-714, doi: 10.1002/hbm.20212.
Fazio, R. H. (2001). On the automatic activation of associated evaluations: An overview. Cognition and Emotion, 15(2), 115-141, doi: 10.1080/02699930125908.
Filippi, P., Ocklenburg, S., Bowling, D. L., Heege, L., Güntürkün, O., Newen, A., & De Boer, B. (2017). More than words (and faces): Evidence for a Stroop effect of prosody in emotion word processing. Cognition and Emotion, 31(5), 879-891, doi: 10.1080/02699931.2016.1177489.
Fox, E., Lester, V., Russo, R., Bowles, R. J., Pichler, A., & Dutton, K. (2000). Facial expressions of emotion: Are angry faces detected more efficiently? Cognition and Emotion, 14(1), 61-92, doi: 10.1080/026999300378996.
Hermans, D., Spruyt, A., & Eelen, P. (2003). Automatic affective priming of recently acquired stimulus valence: Priming at SOA 300 but not at SOA 1000. Cognition and Emotion, 17(1), 83-99, doi: 10.1080/02699930302276.
Kazanas, S. A., & Altarriba, J. (2016). Emotion word processing: Effects of word type and valence in Spanish-English bilinguals. Journal of Psycholinguistic Research, 45(2), 395-406, doi: 10.1007/s10936-015-9357-3.
Kissler, J., Herbert, C., Winkler, I., & Junghofer, M. (2009). Emotion and attention in visual word processing—An ERP study. Biological Psychology, 80(1), 75-83, doi: 10.1016/j.biopsycho.2008.03.004.
Klasen, M., Kreifelts, B., Chen, Y. H., Seubert, J., & Mathiak, K. (2014). Neural processing of emotion in multimodal settings. Frontiers in Human Neuroscience, 8, 822.
Leppänen, J. M., & Hietanen, J. K. (2004). Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research, 69(1-2), 22-29.
Logeswaran, N., & Bhattacharya, J. (2009). Crossmodal transfer of emotion by music. Neuroscience Letters, 455(2), 129-133, doi: 10.1016/j.neulet.2009.03.044.
Paulmann, S., & Pell, M. D. (2010). Contextual influences of emotional speech prosody on face processing: How much is enough? Cognitive, Affective, & Behavioral Neuroscience, 10(2), 230-242.
Pell, M. D. (2005). Nonverbal emotion priming: Evidence from the ‘facial affect decision task’. Journal of Nonverbal Behavior, 29(1), 45-73, doi: 10.1007/s10919-004-0889-8.
Pourtois, G., De Gelder, B., Rossion, B., Vroomen, J., & Crommelinck, M. (2000). The time-course of intermodal binding between seeing and hearing affective information. Neuroreport, 11(6), 1329-1333, doi: 10.1097/00001756-200004270-00036.
Pourtois, G., De Gelder, B., Bol, A., & Crommelinck, M. (2005). Perception of facial expressions and voices and of their combination in the human brain. Cortex, 41(1), 49-59, doi: 10.1016/S0010-9452(08)70177-1.
Segalowitz, N., Trofimovich, P., Gatbonton, E., & Sokolovskaya, A. (2008). Feeling affect in a second language: The role of word recognition automaticity. Mental Lexicon, 3(1), 47-71, doi: 10.1075/ml.3.1.05seg.
Sutton, T. M., Altarriba, J., Gianico, J. L., & Basnight-Brown, D. M. (2007). The automatic access of emotion: Emotional Stroop effects in Spanish-English bilingual speakers. Cognition and Emotion, 21(5), 1077-1090, doi: 10.1080/02699930601054133.
Vaish, A., Grossmann, T., & Woodward, A. (2008). Not all emotions are created equal: The negativity bias in social-emotional development. Psychological Bulletin, 134(3), 383-403, doi: 10.1037/0033-2909.134.3.383.
Vesker, M., Bahn, D., Kauschke, C., Tschense, M., Degé, F., & Schwarzer, G. (2018). Auditory emotion word primes influence emotional face categorization in children and adults, but not vice versa. Frontiers in Psychology, 9, 618, doi: 10.3389/fpsyg.2018.00618.
Vroomen, J., Driver, J., & De Gelder, B. (2001). Is cross-modal integration of emotional expressions independent of attentional resources? Cognitive, Affective, & Behavioral Neuroscience, 1(4), 382-387.
Zinchenko, A., Kanske, P., Obermeier, C., Schröger, E., & Kotz, S. A. (2015). Emotion and goal-directed behavior: ERP evidence on cognitive and emotional conflict. Social Cognitive and Affective Neuroscience, 10(11), 1577-1587, doi: 10.1093/scan/nsv050.

[1]	ZHAO Siyi, LI Ying, WANG Yue. The Metaphorical Association Between Valence and Space Influences Emotional Word Learning in Adults and Children [J]. Studies of Psychology and Behavior, 2022, 20(2): 153-159.
[2]	REN Yu, HAO Fang. The Influence of Emotional Perception and Interpersonal Distance on Altruistic Option [J]. Studies of Psychology and Behavior, 2018, 16(3): 321-326.