Temporal Processing of Rhythm in Speech: Evidence from a Meta-Analysis

doi:10.12139/j.1672-0628.2022.05.005

Abstract

Abstract: A systematic review was conducted to study the time course of speech rhythm and the factors affecting the speech rhythmic effects in the present study. Thirty-six studies and 43 independent effect sizes that met the inclusion criteria of the meta-analysis were selected. The meta-analysis showed that: 1) Rhythm influenced language processing mainly on the N100 and P200 amplitude at the early stages and on the P300, LAN, N400, P600, and LPC amplitude at the middle and late stages. 2) People were very sensitive to the violation of speech rhythm and the processing of speech rhythm preceded syntactic processing. Furthermore, in the middle and late stages, rhythm interacted with semantics and syntax. 3) The moderator analysis revealed that task requirements influenced the rhythmic effect. Specifically, the rhythmic effect was larger in explicit tasks than in implicit and mixed tasks. Attention and anticipation were the intrinsic mechanisms of speech rhythm. Findings support the interactive approach to language processing.

Key words: rhythm, interactive approach, attention, anticipation

摘要： 利用元分析考察了言语节奏在言语加工中起作用的时间进程及影响因素。对36篇脑电文献（共43个独立效应量）进行分析，发现节奏影响言语加工主要表现在早期阶段的N100和P200成分、中后期阶段的P300、LAN、N400和晚期正成分上；个体对节奏违反非常敏感，言语节奏加工早于句法加工，在言语加工的中后期阶段，节奏与语义和句法发生交互；任务类型调节言语节奏对言语加工过程的影响，在外显任务中，言语节奏对言语加工过程的影响大于内隐任务和同时包含多项任务要求的混合任务。注意和预期是言语节奏加工的内在机制，研究结果支持交互论的观点。

关键词: 节奏, 交互论, 注意, 预期

CLC Number:

B842

YAN Qin, ZHANG Qingfang. Temporal Processing of Rhythm in Speech: Evidence from a Meta-Analysis[J]. Studies of Psychology and Behavior, 2022, 20(5): 607-614.

晏芹, 张清芳. 节奏在言语加工中的时间进程：来自元分析的证据[J]. 心理与行为研究, 2022, 20(5): 607-614.

References

方岚, 郑苑仪, 金晗, 李晓庆, 杨玉芳, 王瑞明. (2021). 口语句子的韵律边界: 窥探言语理解的秘窗. 心理科学进展, 29(3), 425–437
冯胜利. (2002). 韵律构词与韵律句法之间的交互作用. 中国语文, (6), 515–524
*罗一丽. (2018). 韵律对汉语词汇、短语和句子加工影响的研究——来自ERPs的证据 (博士学位论文). 南京师范大学.
*罗颖艺. (2012). 汉语句子理解中节奏模式加工的神经机制 (博士学位论文). 北京大学.
徐晓东, 刘昌. (2008). 句子理解的关键——对句法和语义关系的再探讨. 心理科学进展, 16(4), 532–540
*于泽. (2011). 书面韵律信息的作用及其加工机制的即时加工研究 (博士学位论文). 辽宁师范大学, 大连.
*张辉, 孙和涛, 顾介鑫. (2012). 非成语四字格词组加工中韵律与句法互动的ERP研究——熟语表征和理解的认知研究之六. 外语与外语教学, (6), 6–11
*张辉, 孙和涛, 顾介鑫. (2013). 成语加工中韵律与句法互动的事件相关电位研究. 外国语, 36(1), 22–31
*张政华, 韩梅, 张放, 李卫君. (2020). 音乐训练促进诗句韵律整合加工的神经过程. 心理学报, 52(7), 847–860
Beier, E. J., & Ferreira, F. (2018). The temporal prediction of stress in speech and its relation to musical beat perception. Frontiers in Psychology, 9, 431.
*Böcker, K. B. E., Bastiaansen, M. C. M., Vroomen, J., Brunia, C. H. M., & Gelder, B. (1999). An ERP correlate of metrical stress in spoken word recognition. Psychophysiology, 36(6), 706–720.
*Bohn, K., Knaus, J., Wiese, R., & Domahs, U. (2013). The influence of rhythmic (ir)regularities on speech processing: Evidence from an ERP study on German phrases. Neuropsychologia, 51(4), 760–771.
Bouwer, F. L., & Honing, H. (2015). Temporal attending and prediction influence the perception of metrical rhythm: Evidence from reaction times and ERPs. Frontiers in Psychology, 6, 1094.
*Breen, M., Fitzroy, A. B., & Oraa Ali, M. (2019). Event-related potential evidence of implicit metric structure during silent reading. Brain Sciences, 9(8), 192, doi: 10.3390/brainsci9080192.
*Broś, K., Meyer, M., Kliesch, M., & Dellwo, V. (2021). Word stress processing integrates phonological abstraction with lexical access—An ERP study. Journal of Neurolinguistics, 57, 100959.
Canette, L. H., Fiveash, A., Krzonowski, J., Corneyllie, A., Lalitte, P., Thompson, D., .. Tillmann, B. (2020). Regular rhythmic primes boost P600 in grammatical error processing in dyslexic adults and matched controls. Neuropsychologia, 138, 107324, doi: 10.1016/j.neuropsy-chologia.2019.107324.
*Cason, N., & Schön, D. (2012). Rhythmic priming enhances the phonological processing of speech. Neuropsychologia, 50(11), 2652–2658.
*Chen, Q. R., Zhang, J. J., Xu, X. D., Scheepers, C., Yang, Y. M., & Tanenhaus, M. K. (2016). Prosodic expectations in silent reading: ERP evidence from rhyme scheme and semantic congruence in classic Chinese poems. Cognition, 154, 11–21.
Clahsen, H., & Felser, C. (2018). Some notes on the shallow structure hypothesis. Studies in Second Language Acquisition, 40(3), 693–706, doi: 10.1017/S0272263117000250.
Clifton, C., Jr., & Duffy, S. A. (2001). Sentence and text comprehension: Roles of linguistic structure. Annual Review of Psychology, 52, 167–196, doi: 10.1146/annurev.psych.52.1.167.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
*Domahs, U., Genc, S., Knaus, J., Wiese, R., & Kabak, B. (2013). Processing (un-)predictable word stress: ERP evidence from Turkish. Language and Cognitive Processes, 28(3), 335–354.
*Domahs, U., Knaus, J. A., El Shanawany, H., & Wiese, R. (2014). The role of predictability and structure in word stress processing: An ERP study on Cairene Arabic and a cross-linguistic comparison. Frontiers in Psychology, 5, 1151.
*Domahs, U., Knaus, J., Orzechowska, P., & Wiese, R. (2012). Stress “deafness” in a language with fixed word stress: An ERP study on polish. Frontiers in Psychology, 3, 439.
*Domahs, U., Wiese, R., Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2008). The processing of German word stress: Evidence for the prosodic hierarchy. Phonology, 25(1), 1–36.
Eckstein, K., & Friederici, A. D. (2005). Late interaction of syntactic and prosodic processes in sentence comprehension as revealed by ERPs. Cognitive Brain Research, 25(1), 130–143, doi: 10.1016/j.cogbrainres.2005.05.003.
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634, doi: 10.1136/bmj.315.7109.629.
Emmendorfer, A. K., Correia, J. M., Jansma, B. M., Kotz, S. A., & Bonte, M. (2020). ERP mismatch response to phonological and temporal regularities in speech. Scientific Reports, 10(1), 9917, doi: 10.1038/s41598-020-66824-x.
Fodor, J. D. (2002). Psycholinguistics cannot escape prosody. In Proceedings of the Speech Prosody (pp. 83–88). Aix-en Provence, France: Laboratoire Parole et Langage.
Fotidzis, T. S., Moon, H., Steele, J. R., & Magne, C. L. (2018). Cross-modal priming effect of rhythm on visual word recognition and its relationships to music aptitude and reading achievement. Brain Sciences, 8(12), 210, doi: 10.3390/brainsci8120210.
Friederici, A. D. (2011). The brain basis of language processing: From structure to function. Physiological Reviews, 91(4), 1357–1392, doi: 10.1152/physrev.00006.2011.
Gasparini, L., Langus, A., Tsuji, S., & Boll-Avetisyan, N. (2021). Quantifying the role of rhythm in infants’ language discrimination abilities: A meta-analysis. Cognition, 213, 104757, doi: 10.1016/j.cognition.2021.104757.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.
*Henrich, K., Alter, K., Wiese, R., & Domahs, U. (2014). The relevance of rhythmical alternation in language processing: An ERP study on English compounds. Brain and Language, 136, 19–30.
*Henrich, K., Wiese, R., & Domahs, U. (2015a). The influence of syllable number and task-related attention on the perception of rhythmic irregularities: An ERP study on German compounds. Paper presented at the 18th International Conference on Phonetic Sciences, Glasgow, United Kingdom.
*Henrich, K., Wiese, R., & Domahs, U. (2015b). How information structure influences the processing of rhythmic irregularities: ERP evidence from German phrases. Neuropsychologia, 75, 431–440.
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560, doi: 10.1136/bmj.327.7414.557.
Holcomb, P. J., Coffey, S. A., & Neville, H. J. (1992). Visual and auditory sentence processing: A developmental analysis using event-related brain potentials. Developmental Neuropsychology, 8(2–3), 203–241.
Jones, A. (2019). Temporal expectancies and rhythmic cueing in touch: The influence of spatial attention. Cognition, 182, 140–150, doi: 10.1016/j.cognition.2018.09.011.
*Knaus, J., Wiese, R., & Janßen, U. (2007). The processing of word stress: EEG studies on task-related components. In J. Trouvain & W. J. Barry (Eds.), Proceedings of the 16th International Conference of Phonetic Sciences (pp. 709–712). Saarbrücken, Germany: Universitaet des Saarlandes.
*Koso, A., & Hagiwara, H. (2009). Event-related potential evidence of processing lexical pitch-accent in auditory Japanese sentences. Neuroreport, 20(14), 1270–1274.
*Koso, A., Ojima, S., & Hagiwara, H. (2011). An event-related potential investigation of lexical pitch-accent processing in auditory Japanese. Brain Research, 1385, 217–228.
*Kriukova, O., & Mani, N. (2016). Processing metrical information in silent reading: An ERP study. Frontiers in Psychology, 7, 1432.
Kubozono, H. (2015). Introduction to Japanese phonetics and phonology. In H. Kubozono (Ed.), Handbook of Japanese phonetics and phonology (pp. 1–42). Boston: De Gruyter Mouton.
Large, E. W., & Jones, M. R. (1999). The dynamics of attending: How people track time-varying events. Psychological Review, 106(1), 119–159, doi: 10.1037/0033-295X.106.1.119.
Lee, C. S., & Todd, N. P. M. (2004). Towards an auditory account of speech rhythm: Application of a model of the auditory ‘primal sketch’ to two multi-language corpora. Cognition, 93(3), 225–254, doi: 10.1016/j.cognition.2003.10.012.
*Lee, Y., Liu, X., & Wang, X. M. (2015). Do metrical incongruities disrupt semantic processing in modern Chinese? An event-related potential study. NeuroReport, 26(14), 803–808.
Li, X. Q., Shao, X. M., Xia, J. Y., & Xu, X. Y. (2019). The cognitive and neural oscillatory mechanisms underlying the facilitating effect of rhythm regularity on speech comprehension. Journal of Neurolinguistics, 49, 155–167, doi: 10.1016/j.jneuroling.2018.05.004.
Luo, C., & Ding, N. (2020). Cortical encoding of acoustic and linguistic rhythms in spoken narratives. eLife, 9, e60433, doi: 10.7554/eLife.60433.
*Luo, Y. Y., & Zhou, X. L. (2010). ERP evidence for the online processing of rhythmic pattern during Chinese sentence reading. NeuroImage, 49(3), 2836–2849.
*Magne, C., Astésano, C., Aramaki, M., Ystad, S., Kronland-Martinet, R., & Besson, M. (2007). Influence of syllabic lengthening on semantic processing in spoken French: Behavioral and electrophysiological evidence. Cerebral Cortex, 17(11), 2659–2668.
*Magne, C., Gordon, R. L., & Midha, S. (2010). Influence of metrical expectancy on reading words: An ERP study. In Proceedings of the Fifth International Conference on Speech Prosody(p. 432). Chicago.
Magne, C., Jordan, D. K., & Gordon, R. L. (2016). Speech rhythm sensitivity and musical aptitude: ERPs and individual differences. Brain and Language, 153–154, 13–19.
*Marie, C., Magne, C., & Besson, M. (2011). Musicians and the metric structure of words. Journal of Cognitive Neuroscience, 23(2), 294–305.
*McCauley, S. M., Hestvik, A., & Vogel, I. (2013). Perception and bias in the processing of compound versus phrasal stress: Evidence from event-related brain potentials. Language and Speech, 56(1), 23–44.
Miyamoto, T., & Johnson, C. (2002). Accentual phrasing in Japanese: The significance of underlying accents. In Proceedings of the Speech Prosody (pp. 519–522). Aix-en-Provence, France: Laboratoire Parole et Langage.
*Mołczanow, J., Domahs, U., Knaus, J., & Wiese, R. (2013). The lexical representation of word stress in Russian: Evidence from event-related potentials. The Mental Lexicon, 8(2), 164–194.
Nolan, F., & Asu, E. L. (2009). The Pairwise Variability Index and coexisting rhythms in language. Phonetica, 66(1–2), 64–77.
Poeppel, D., & Assaneo, M. F. (2020). Speech rhythms and their neural foundations. Nature Reviews Neuroscience, 21(6), 322–334, doi: 10.1038/s41583-020-0304-4.
Polich, J. (2007). Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology, 118(10), 2128–2148, doi: 10.1016/j.clinph.2007.04.019.
Rothermich, K., & Kotz, S. A. (2013). Predictions in speech comprehension: fMRI evidence on the meter-semantic interface. NeuroImage, 70, 89–100, doi: 10.1016/j.neuroimage.2012.12.013.
*Rothermich, K., Schmidt-Kassow, M., & Kotz, S. A. (2012). Rhythm’s gonna get you: Regular meter facilitates semantic sentence processing. Neuropsychologia, 50(2), 232–244.
*Rothermich, K., Schmidt-Kassow, M., Schwartze, M., & Kotz, S. A. (2010). Event-related potential responses to metric violations: Rules versus meaning. NeuroReport, 21(8), 580–584.
Roncaglia-Denissen, M. P., Schmidt-Kassow, M., & Kotz, S. A. (2013). Speech rhythm facilitates syntactic ambiguity resolution: ERP evidence. PLoS One, 8(2), e56000, doi: 10.1371/journal.pone.0056000.
Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. Science, 274(5294), 1926–1928, doi: 10.1126/science.274.5294.1926.
Schmidt-Kassow, M., & Kotz, S. A. (2008). Entrainment of syntactic processing? ERP-responses to predictable time intervals during syntactic reanalysis. Brain Research, 1226, 144–155, doi: 10.1016/j.brainres.2008.06.017.
*Schmidt-Kassow, M., & Kotz, S. A. (2009a). Event-related brain potentials suggest a late interaction of meter and syntax in the P600. Journal of Cognitive Neuroscience, 21(9), 1693–1708.
*Schmidt-Kassow, M., & Kotz, S. A. (2009b). Attention and perceptual regularity in speech. NeuroReport, 20(18), 1643–1647.
*Schmidt-Kassow, M., Roncaglia-Denissen, M. P., & Kotz, S. A. (2011). Why pitch sensitivity matters: Event-related potential evidence of metric and syntactic violation detection among Spanish late learners of German. Frontiers in Psychology, 2, 131.
Snedeker, J., & Trueswell, J. (2003). Using prosody to avoid ambiguity: Effects of speaker awareness and referential context. Journal of Memory and Language, 48(1), 103–130, doi: 10.1016/S0749-596X(02)00519-3.
Takacs, Z. K., & Kassai, R. (2019). The efficacy of different interventions to foster children’s executive function skills: A series of meta-analyses. Psychological Bulletin, 145(7), 653–697, doi: 10.1037/bul0000195.
Teki, S., & Griffiths, T. D. (2014). Working memory for time intervals in auditory rhythmic sequences. Frontiers in Psychology, 5, 1329.
Tooley, K. M., Konopka, A. E., & Watson, D. G. (2018). Assessing priming for prosodic representations: Speaking rate, intonational phrase boundaries, and pitch accenting. Memory & Cognition, 46(4), 625–641.
Yao, B., Belin, P., & Scheepers, C. (2011). Silent reading of direct versus indirect speech activates voice-selective areas in the auditory cortex. Journal of Cognitive Neuroscience, 23(10), 3146–3152, doi: 10.1162/jocn_a_00022.
*Zhang, N., & Zhang, Q. F. (2019). Rhythmic pattern facilitates speech production: An ERP study. Scientific Reports, 9(1), 12974.
Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G. M., … Schroeder, C. E. (2013). Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”. Neuron, 77(5), 980–991, doi: 10.1016/j.neuron.2012.12.037.

[1]	LI Shiyi, XIE Yanfeng, ZHAO Guang, BAI Xuejun. The Influence of Media Multitasking Experience on Implicit Memory in Different Attention Patterns [J]. Studies of Psychology and Behavior, 2022, 20(4): 433-440.
[2]	LIU Yanyan, LI Jie, ZHAO Qichao, CHEN Beibei, ZHANG Yu. Diffusion of Inhibition of Return in 3D Dynamic and Static Scenes [J]. Studies of Psychology and Behavior, 2022, 20(1): 22-28.
[3]	SU Mingyu, WANG Guangming. STEAM Training Can Improve Pupils’ Attentional Inhibition and Attentional Concentration Ability [J]. Studies of Psychology and Behavior, 2021, 19(5): 701-705.
[4]	WU-REN Qimuge, ZHAO Aoxue, WANG Yuhan, ZHUANG Xiangling, MA Guojie. Visual Field Preference and Reference Strategy in Comparative Visual Search [J]. Studies of Psychology and Behavior, 2021, 19(2): 166-171.
[5]	GAO Chao, LI Shiyi, ZHANG Peng, ZHAO Guang. Modulation of Depth Perception on Contextual Cueing Effect Based on Binocular Disparity [J]. Studies of Psychology and Behavior, 2021, 19(1): 16-21,29.
[6]	JIAO Jiangli, LIU Yi, WEN Suxia. Bilingual Advantage of Bilingualism in Emotional Attention Bias [J]. Studies of Psychology and Behavior, 2020, 18(5): 590-595.
[7]	TIAN Ye, HE Luning, LIU Tianjiao, CHANG Ruosong, MA Jinfei. Effects of Interesting Auditory Materials on Driving Fatigue: Evidence from EEG [J]. Studies of Psychology and Behavior, 2020, 18(4): 474-481.
[8]	CHE Xiaowei, WANG Kaixuan, SHANGGUAN Mengqi, LI Shouxin. The Representation of Attention Template in Visual Working Memory: An EROS Study [J]. Studies of Psychology and Behavior, 2020, 18(3): 297-303.
[9]	WU Huifen, LI Xiaona, ZHANG Qiong, TANG Yichun, LIU Huashan. You Own Something Because of Paying Attention to Them: Exploring the Internal Mechanism of Mere Ownership Effect [J]. Studies of Psychology and Behavior, 2020, 18(2): 161-167,175.
[10]	LI Hengping, HE Shengxi, LIU Zhengkui. ERPs Study on the Effect of Fast Reading Training on Attention Based on Visual Beam Search [J]. Studies of Psychology and Behavior, 2020, 18(1): 78-84.
[11]	YU Yongju. Effects of Anxiety and Depression on Attentional Bias: The Mediating Role of Attentional Control [J]. Studies of Psychology and Behavior, 2020, 18(1): 121-127.
[12]	ZENG Qing, ZHENG XiFu. The Time-course Difference of the Attention Paid by High Disgust Sensitivity and Low Disgust Sensitivity Individuals when Processing Different Emotional Pictures: An Event-related Potentials Study [J]. Studies of Psychology and Behavior, 2019, 17(6): 743-749.
[13]	WANG Tingzhao, WANG Fengli, AO Yongqian, WANG Li. A Comparative Study on the Effects of Emotional Stimulation on Attention Blink in Deaf and Normal-Hearing People under the RSVP Paradigm [J]. Studies of Psychology and Behavior, 2019, 17(6): 803-809.
[14]	WU Lei, ZHAO Boyao, ZHAN Zhao, FAN Bijuan, ZHAO Yaping. The Effect of Mindful Parenting from Parents on Adolescent Depression and its Mechanism [J]. Studies of Psychology and Behavior, 2019, 17(4): 561-568.
[15]	LIU Li, LI Shiyi, YUE Junbing, BAI Xuejun. Location-value Association Induces Attentional Capture and Underlying Mechanism [J]. Studies of Psychology and Behavior, 2019, 17(3): 289-295.