The Influence of Speech-Associated Gestures on Cognitive Load in English Multimedia Learning

Abstract

Abstract: To explore the influence of speech-associated gestures on cognitive load and performance in English multimedia learning, the research used a 2×2 design with two between-subjects factors being level of English language skills and multimedia presentation of language learning material. The results showed that: 1)there was no main effect of speech-associated gestures on cognitive load, but there was interaction effect between the gestures and the level of English language skills. When the level of English language skills was high, the gestures decreased cognitive load; otherwise, they increased it. 2)The speech-associated gestures had no significant influence on the performance of sentence transformation, but they could increase that of comprehension ability when the level of English language skills was high. The findings suggest that the utilization of speech-associated gestures has an obvious effect in English multimedia learning, which can increase or decrease cognitive load of learners, and has an influence on their performance of comprehension ability. However, degree and direction of the effect depend on the level of learners′ English language skills.

Key words: speech-associated gesture, cognitive load, multimedia learning, instructional design.

摘要： 为了探讨英语多媒体学习中言语关联手势对认知负荷及学习成绩的影响，采用2×2被试间实验设计。结果发现：该手势对认知负荷影响的主效应不显著，但与英语语言技能水平之间存在交互作用，当学生语言技能水平低时，手势增加了认知负荷，反之，则降低认知负荷；该手势对句子转换的成绩没有明显影响，但在学生语言技能高时能提升理解能力的成绩。本研究结果提示，言语关联手势的运用有明显作用，它能提高或降低认知负荷，并对理解能力的成绩产生影响，但其大小和方向依赖于学生英语语言技能水平。

关键词: 言语关联手势, 认知负荷, 多媒体学习, 教学设计。

CLC Number:

B842

Sun Chongyong. The Influence of Speech-Associated Gestures on Cognitive Load in English Multimedia Learning[J]. Studies of Psychology and Behavior, 2016, 14(5): 633-639.

孙崇勇. 英语多媒体学习中言语关联手势对认知负荷的影响[J]. 心理与行为研究, 2016, 14(5): 633-639.

References

[1] 白学军, 梁菲菲, 张涛, 田丽娟, 文宇翔, 陈宗阳.(2009).不同获奖等级青年教师手势语的量化研究. 宁波大学学报(教育科学版), 31(4), 48-53.
[2] 陈桦, 顾琰, 左瀚.(2013). 手势与言语的关系论. 外国语, 36(5), 50-58.
[3] 段朝辉, 颜志强, 王福兴, 周宗奎.(2013). 动画呈现速度对多媒体学习效果影响的眼动研究. 心理发展与教育, 29(1), 46-53.
[4] 李智晔.(2013). 多媒体学习的认知—传播模型及其基本特征. 教育研究, 35(8), 112-116.
[5] 杨晓琼.(2011). 国外手势与二语习得研究的进展. 外国语文, 27(3), 89-93.
[6] 赵立影.(2011). 基于工作记忆的多媒体学习设计. 电化教育研究, 32(8), 98-102.
[7] 朱明泉, 张智君.(2007). 言语与手部运动关系的研究回顾. 心理科学进展, 15(1), 88-93.
[8] Barbieri, F., Buonocore, A., Volta, R. D., & Gentilucci, M.(2009). How symbolic gestures and words interact with each other. Brain and Language, 110(1), 1-11.
[9] Boucheix, S. C., & Lowe, R. K.(2010). An eye tracking comparison of external pointing cues and internal continuous cues in learning with complex animations. Learning and Instruction, 20(2), 123-135.
[10] Churchill, E., Okada, H., Nishino, T., & Atkinson, D.(2010). Symbiotic gesture and the sociocognitive visibility of grammar in second language acquisition. The Modern Language Journal, 94(2), 234-253.
[11] Cierniak, G., Scheiter, K., & Gerjiets, P.(2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load?. Computers in Human Behavior, 25(2), 315-324.
[12] De Koning, B. B., & Tabbers, H. K.(2011). Facilitating understanding of movements in dynamic visualizations: An embodied perspective. Educational Psychology Review, 23(4), 501-521.
[13] De Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Pass, F.(2009). Towards a framework for attention cueing in instructional animations: Guidelines for research and design. Educational Psychology Review, 21(2), 113-140.
[14] De Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Pass, F.(2010a). Learning by generating vs. Receiving instructional explanations: Two approaches to enhance attention cueing in animations. Computers & Education, 55(2), 681-691.
[15] De Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Pass, F.(2010b). Attention guidance in learning from a complex animation: Seeing is understanding?. Learning and instruction, 20(2), 111-122.
[16] DeLeeuw, K. E., & Mayer, R. E.(2008). A comparison of three measures of cognitive load, evidence for separable measures of intrinsic, extraneous, and germane load. Journal of Educational Psychology, 100(1), 223-234.
[17] Flevares, L. M., & Perry, M.(2001). How many do you see? The use of nonspoken representations in first grade mathematics lessons. Journal of Educational Psychology, 93(2), 330–345.
[18] Goldin-Meadow, S., & Butcher, C.(2003). Pointing: Where language, culture, and cognition meet. In S. Kita, Pointing toward two word speech in young children(pp. 85-107). Hillsdale NJ: Lawrence Erlbaum Associates.
[19] Goldin-Meadow, S., Nusbaum, H., Garber, P., & Church, R. B.(1993). Transitions in learning: Evidence for simultaneously activated strategies. Journal of Experimental Psychology: Human Perception and Performance, 19(1), 91-107.
[20] Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., & Wagner, S.(2001). Explaining math: Gesturing lightens the load. Psychological Science, 12(6), 516-522.
[21] Kalyuaga, S.(2011). Cognitive load theory: How many types of load does it really need?. Educational Psychology Review, 23(1), 1-19.
[22] Kelly, S. D., Manning, S. M., & Rodak, S.(2008). Gesture gives a hand to language and learning, Perspectives from cognitive neuroscience, developmental psychology and education. Language and Linguistics Compass, 2(4), 569-588.
[23] Mayer, R. E., & Moreno, R.(1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312-330.
[24] Morrison, J. B., Tversky, B., & Betrancourt, M.(2000). Animation: Does it facilitate learning. Paper presented at the 2000 AAAI Spring Symposia, Palo Alto, California.
[25] Paas, F., & Van Merrienboer, J. J. G.(1993).The efficiency of instructional conditions: An approach to combine mental effort and performance measures. Human Factors, 35(4), 737-743.
[26] Paas, F., Renkl, A., & Sweller, J.(2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1-4.
[27] Ping, R. M., & Goldin-Meadow，S.(2008). Hands in the air: Using ungrounded iconic gestures to teach children conservation of quantity. Development Psychology, 44(5), 1277-1287.
[28] Skipper, J. I., Goldin-Meadow, S., Nusbaum, H. C., & Small, S. L.(2007). Speech-associated gestures, Broca′s area，and the human mirror system. Brain and Language, 101(3), 260-277.
[29] Spanjers, I. A. E., Van Gog, T., Wouters, P., & Van Merrienboer, J. J. G.(2012). Explaining the segmentation effect in learning from animations: The role of pausing and temporal cueing. Computer & Education, 59(2), 274-280.
[30] Sweller, J.(1994). Cognitive load theory, learning difficulty, an instructional design. Learning and Instruction, 4(4), 295-312.
[31] Sweller, J.(1988). Cognitive load during problem solving, effects on learning. Cognitive Science, 12(2), 257-285.
[32] Sweller, J., Van Merrienboer, J. J. G., & Paas, F.(1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296.
[33] Wong, A., Marcus, N., Ayres, P., Smith, L., Copper, G. A., & Paas, F., et al.(2009). Instruction animations can be superior to statics when learning human motor skills. Computers in Human Behavior, 25(2), 339-347.
[34] Yoshioka, K., & Kellerman, E.(2006). Gesture Introduction of Ground Reference in L2 Narrative Discourse. International Review of Applied Linguistics, 44(2), 171-193.