Probabilistic Linguistic Expectations, Uncertain Input, and Implications for Eye Movements in Reading

摘要/Abstract

摘要： One nearly ubiquitous assumption in models of linguistic comprehension and of eye movement control in reading alike is of partial modularization between word-level and sentence-level processing: that the outcome of word recognition, and thus the input to sentence-level comprehension, is a categorial representation. Yet such a partial modularization throws away residual uncertainty regarding word identity that might potentially be of value to the comprehender further downstream in the sentence. Here I describe a line of research combining computational modeling with experimental eye-tracking work to explore the consequences of removing this partial modularity assumption.

Abstract: One nearly ubiquitous assumption in models of linguistic comprehension and of eye movement control in reading alike is of partial modularization between word-level and sentence-level processing: that the outcome of word recognition, and thus the input to sentence-level comprehension, is a categorial representation. Yet such a partial modularization throws away residual uncertainty regarding word identity that might potentially be of value to the comprehender further downstream in the sentence. Here I describe a line of research combining computational modeling with experimental eye-tracking work to explore the consequences of removing this partial modularity assumption.

Roger Levy. Probabilistic Linguistic Expectations, Uncertain Input, and Implications for Eye Movements in Reading[J]. 心理与行为研究, 2011, 9(1): 53-64.

Roger Levy. Probabilistic Linguistic Expectations, Uncertain Input, and Implications for Eye Movements in Reading[J]. Studies of Psychology and Behavior, 2011, 9(1): 53-64.

参考文献

[1] Altmann, G. T., & Kamide, Y.(1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73 (3), 247-264.
[2] Bicknell, K., & Levy, R.(2009). A model of local coherence effects in human sentence processing as consequences of updates from bottom-up prior to posterior beliefs. In Proceedings of the north american chapter of the association for computational linguistics-human language technologies(NAACL-HLT)conference.
[3] Bicknell, K., & Levy, R.(2010). A rational model of eye movement control in reading. In Proceedings of the annual meeting of the association for computational linguistics (pp.1168-1178).
[4] Booth, T. L.(1969). Probabilistic representation of formal languages. In Ieee conference record of the 1969 tenth annual symposium on switching and automata theory (pp.74-81).
[5] Chen, S., & Goodman, J.(1998). An empirical study of smoothing techniques for language modeling (Tech. Rep.). Computer Science Group, Harvard University.
[6] Chomsky, N.(1956). Three models for the description of language. IRE Transactions on Information Theory, 2 (3), 113-124.
[7] Church, K., & Patil, R.(1982). Coping with syntactic ambiguity, or how to put the block in the box on the table. Computational Linguistics, 8 (3-4), 139-149.
[8] Cover, T., & Thomas, J.(1991). Elements of information theory . John Wiley.
[9] Crocker, M., & Brants, T.(2000). Wide - coverage probabilistic sentence processing. Journal of Psycholinguistic Research, 29 (6), 647-669.
[10] Earley, J.(1970). An efficient context - free parsing algorithm. Communications of the ACM, 13 (2), 94-102.
[11] Ehrlich, S. F., & Rayner, K.(1981). Contextual effects on word perception and eye movements during reading. Journal of Verbal Learning and Verbal Behavior, 20 , 641-655.
[12] Ford, M., Bresnan, J., & Kaplan, R. M.(1982). A competence - based theory of syntactic closure. In J. Bresnan(Ed.), The mental representation of grammatical relations (pp. 727-796). The MIT Press.
[13] Frazier, L., & Rayner, K.(1982). Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences. Cognitive Psychology, 14 , 178-210.
[14] Gibson, E.(2006). The interaction of top-down and bottom-up statistics in the resolution of syntactic category ambiguity. Journal of Memory and Language, 54 , 363-388.
[15] Ginsburg, S.(1966). The mathematical theory of context - free languages . McGraw - Hill.
[16] Hale, J.(2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of the second meeting of the north american chapter of the association for computational linguistics (pp. 159-166).
[17] Hale, J.(2003). The information conveyed by words in sentences. Journal of Psycholinguistic Research, 32 (2), 101-123.
[18] Hale, J.(2006). Uncertainty about the rest of the sentence. Cognitive Science, 30 (4), 609-642.
[19] Hopcroft, J. E., & Ullman, J. D.(1979). Introduction to automata theory, languages, and computation . Addison - Wesley.
[20] Jelinek, F., & Lafferty, J. D.(1991). Computation of the probability of initial substring 24 generation by stochastic context free grammars. Computational Linguistics, 17 (3), 315-323.
[21] Jurafsky, D.(1996). A probabilistic model of lexical and syntactic access and disambiguation. Cognitive Science, 20 (2), 137-194.
[22] Jurafsky, D., & Martin, J. H.(2008). Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition (Second ed.). Prentice - Hall.
[23] Kamide, Y., Altmann, G., & Haywood, S.(2003). The time - course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language, 49 (1), 133-156.
[24] Kliegl, R., Nuthmann, A., & Engbert, R.(2006). Tracking the mind during reading: The influence of past, present, and future words on fixation durations. Journal of Experimental Psychology: General, 135 (1), 12-35.
1., H., & Francis, W. N.(1967). Computational analysis of present - day American English. Providence, RI: Brown University Press.
[25] Levenshtein, V. I.(1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10 , 707-710.
[26] Levy, R.(2008a). Expectation - based syntactic comprehension. Cognition, 106 , 1126-1177.
[27] Levy, R.(2008b). A noisy - channel model of rational human sentence comprehension under uncertain input. In Proceedings of the 13th conference on empirical methods in natural language processing (pp. 234-243).
[28] Levy, R.(2010, March). On hallucinated garden paths . Poster presented at the CUNY sentence processing conference.
[29] Levy, R., Bicknell, K., Slattery, T., & Rayner, K.(2009). Eye movement evidence that readers maintain and act on uncertainty about past linguistic input. Proceedings of the national academy of sciences, 106 (50), 21086-21090.
[30] Levy, R., Reali, F., & Griffiths, T. L.(2009). Modeling the effects of memory on human online sentence processing with particle filters. In Proceedings of the 22nd conference on neural information processing systems(nips) .
[31] Manning, C. D., & Schütze, H.(1999). Foundations of statistical Kuera natural language processing . MIT Press.
[32] Marcus, M. P., Santorini, B., & Marcinkiewicz, M. A.(1994). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19 (2), 313-330.
[33] Morgan, E., Keller, F., & Steedman, M.(2010). A bottom - up parsing model of local coherence effects. In Proceedings of the cognitive science society conference .
[34] Narayanan, S., & Jurafsky, D.(1998). Bayesian models of human sentence processing. In Proceedings of the twelfth annual meeting of the cognitive science society .
[35] Narayanan, S., & Jurafsky, D.(2002). A Bayesian model predicts human parse preference and reading time in sentence processing. In Advances in neural information processing systems (Vol.14, pp. 59-65).
[36] Norris, D.(2006). The Bayesian Reader: Explaining word recognition as an optimal Bayesian decision process. Psychological Review, 113 (2), 327-357.
[37] Rayner, K.(1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124 (3), 372-422.
[38] Rayner, K.(2009). The thirty-fifth Sir Frederick Bartlett Lecture: Eye movements and attention during reading, scene perception, and visual search. Quarterly Journal of Experimental Psychology, 62 , 1457-1506.
[39] Rayner, K., & Well, A. D.(1996). Effects of contextual constraint on eye movements in reading: A further examination. Psychonomic Bulletin & Review, 3 (4), 504-509.
[40] Roark, B.(2001). Probabilistic top-down parsing and language modeling. Computational Linguistics, 27 (2), 249-276.
[41] Russell, S., & Norvig, P.(2003). Artificial intelligence: A modern approach (Second ed.). Prentice Hall.
[42] Smith, N. J., & Levy, R.(2008). Optimal processing times in reading: A formal model and empirical investigation. In Proceedings of the 30th annual meeting of the cognitive science society.
[43] Smith, N. J., & Levy, R.(2010). Fixation durations in first-pass reading reflect uncertainty about word identity. In Proceedings of the 32nd annual meeting of the cognitive science society.
[44] Smith, N. J., & Levy, R.(Submitted). A rational model of predictability effects on cognitive processing times .(Manuscript, UC San Diego).
[45] Stolcke, A.(1995). An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics, 21 (2), 165-201.
[46] Sutton, R. S., & Barto, A. G.(1998). Reinforcement learning. MIT Press.
[47] Tabor, W., Galantucci, B., & Richardson, D.(2004). Effects of merely local syntactic coherence on sentence processing. Journal of Memory and Language, 50 (4), 355-370.
[48] Tabor, W., & Hutchins, S.(2004). Evidence for self - organized sentence processing: Digging in effects. Journal of Experimental Psychology: Learning, Memory, & Cognition, 30 (2), 431-450.
[49] Tanenhaus, M. K., Spivey - Knowlton, M. J., Eberhard, K., & Sedivy, J. C.(1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268 , 1632-1634.
[50] Taylor, W. L.(1953). A new tool for measuring readability. Journalism Quarterly, 30 , 415.