The Trade-off between Speed and Accuracy: Evaluation and Modeling for Participants in the Response States

Abstract

Abstract: Most tests are administered within an allocated time. Within the limited time, participants might have different trade-offs on different items. With the increase of response time, the accuracy reduces when it is improved to a certain point in some items. In view of this situation, this study explored the causes from the point of view of ability, speed, ability and speed compensation. Therefore, three hierarchical models of the non-monotonic relationship between response time and accuracy were constructed. The results obtained in the simulation studies indicated that the MCMC algorithm performed well and enabled simultaneous estimation. More specifically, increasing the sample size of the participants led to better estimation for item parameters; and increasing the test length, the person parameters became better. Finally, three proposed hierarchical models have been empirically studied by a real-data example in the Raven's Standard Progressive Matrices. The results showed that the non-monotonic relationship between response time and accuracy is more likely to be affected by the ability level.

Key words: response time, accuracy, the non-monotonic relationship, hierarchical model

摘要： 在测验中，被试总是被要求在规定的时间内完成测验作答。在测验时间限制内，被试在不同项目上可能存在不同的权衡倾向。在有的项目上，随着反应时增加，准确率提高到一定程度后反而会降低。针对这种情形，本研究从能力、速度以及能力与速度补偿角度对其原因进行探索，并以此构建了三个反应时与准确率非单调关系的分层模型。结果表明，MCMC（Markov chain Monte Carlo）算法能准确地估计模型各参数；而且在实际数据分析中发现反应时与准确率的非单调关系更可能与被试的能力有关。

关键词: 反应时, 准确率, 非单调关系, 分层模型

CLC Number:

B841

GUO Xiaojun, LUO Zhaosheng. The Trade-off between Speed and Accuracy: Evaluation and Modeling for Participants in the Response States[J]. Studies of Psychology and Behavior, 2019, 17(5): 589-595.

郭小军, 罗照盛. 速度与准确率权衡：被试反应状态评价与建模[J]. 心理与行为研究, 2019, 17(5): 589-595.

References

孟祥斌. (2016). 项目反应时间的对数偏正态模型. 心理科学, 39(3), 727-734
朱滢. (2009). 实验心理学(第2版). 北京:北京大学出版社.
Anders, R., Alario, F. X., & van Maanen, L. (2016). The shifted Wald distribution for response time data analysis. Psychological Methods, 21(3), 309-327, doi:10.1037/met0000066.
Bolsinova, M., De Boeck, P., & Tijmstra, J. (2017). Modelling conditional dependence between response time and accuracy. Psychometrika, 82(4), 1126-1148, doi:10.1007/s11336-016-9537-6.
Bolsinova, M., Tijmstra, J., & Molenaar, D. (2017). Response moderation models for conditional dependence between response time and response accuracy. British Journal of Mathematical and Statistical Psychology, 70(2), 257-279, doi:10.1111/bmsp.12076.
Brooks, S. P., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434-455.
Chen, H. Q., De Boeck, P., Gradya, M., Yanga, C. L., & Waldschmidta, D. (2018). Curvilinear dependency of response accuracy on response time in cognitive tests. Intelligence, 69, 16-23, doi:10.1016/j.intell.2018.04.001.
Donkin, C., Little, D. R., & Houpt, J. W. (2014). Assessing the speed-accuracy trade-off effect on the capacity of information processing. Journal of Experimental Psychology:Human Perception and Performance, 40(3), 1183-1202, doi:10.1037/a0035947.
Giordano, A. M., McElree, B., & Carrasco, M. (2009). On the automaticity and flexibility of covert attention:A speed-accuracy trade-off analysis. Journal of Vision, 9(3), 1-10, doi:10.1167/9.3.1.
Goldhammer, F., Naumann, J., & Greiff, S. (2015). More is not always better:The relation between item response and item response time in Raven's matrices. Journal of Intelligence, 3(1), 21-40, doi:10.3390/jintelligence3010021.
Grigorenko, E. L., & Sternberg, R. J. (1995). Thinking styles. In D. H. Saklofske & M. Zeidner (Eds.), Perspectives on individual differences. International handbook of personality and intelligence. New York:Plenum Press.
Heathcote, A. (2004). Fitting Wald and ex-Wald distributions to response time data:An example using functions for the S-PLUS package. Behavior Research Methods, Instruments, & Computers, 36(4), 678-694.
Heitz, R. P. (2014). The speed-accuracy tradeoff:History, physiology, methodology, and behavior. Frontiers in Neuroscience, 8, 150.
Kagan, J., Rosman, B. L., Day, D., Albert, J., & Phillips, W. (1964). Information processing in the child:Significance of analytic and reflective attitudes. Psychological Monographs:General and Applied, 78(1), 1-37, doi:10.1037/h0093830.
McElree, B. (1998). Attended and non-attended states in working memory:Accessing categorized structrures. Journal of Memory and Language, 38(2), 225-252, doi:10.1006/jmla.1997.2545.
McElree, B. (2000). Sentence comprehension is mediated by content-addressable memory structures. Journal of Psycholinguistic Research, 29(2), 111-123, doi:10.1023/A:1005184709695.
McElree, B., & Carrasco, M. (1999). The Temporal Dynamics of Visual Search:Evidence for Parallel Processing in Feature and Conjunction Searches. Journal of Experimental Psychology:Human Perception and Performance, 25(6), 1517-1539, doi:10.1037/0096-1523.25.6.1517.
McElree, B., Foraker, S., & Dyer, L. (2003). Memory structures that subserve sentence comprehension. Journal of Memory and Language, 48(1), 67-91, doi:10.1016/S0749-596X(02)00515-6.
McElree, B., Jia, G., & Litvak, A. (2000). the time course of conceptual processing in three bilingual populations. Journal of Memory and Language, 42(2), 229-254, doi:10.1006/jmla.1999.2677.
Meng, X. B., Tao, J., & Chang, H. H. (2015). A conditional joint modeling approach for locally dependent item responses and response times. Journal of Educational Measurement, 52(1), 1-27, doi:10.1111/jedm.12060.
Ratcliff, R., Smith, P. L., Brown, S. D., & McKoon, G. (2016). Diffusion decision model:Current issues and history. Trends in Cognitive Sciences, 20(4), 260-281, doi:10.1016/j.tics.2016.01.007.
Reed, A. V. (1973). Speed-accuracy trade-off in recognition memory. Science, 181(4099), 574-576, doi:10.1126/science.181.4099.574.
Reed, A. V. (1976). List length and the time course of recognition in immediate memory. Memory & Cognition, 4(1), 16-30.
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society. Series B, 64, 583-616, doi:10.1111/1467-9868.00353.
van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287-308, doi:10.1007/s11336-006-1478-z.
van der Linden, W. J. (2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46(3), 247-272, doi:10.1111/j.1745-3984.2009.00080.x.
van der Maas, H. L. J., Molenaar, D., Maris, G., Kievit, R., & Borsboom, D. (2011). Cognitive psychology meets psychometric theory:On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118(2), 339-356, doi:10.1037/a0022749.
Wang, T. Y., & Zhang, J. W. (2006). Optimal partitioning of testing time:Theoretical properties and practical implications. Psychometrika, 71(1), 105-120, doi:10.1007/s11336-002-1013-x.
Wenger, M. J., & Gibson, B. S. (2004). Using hazard functions to assess changes in processing capacity in an attentional cuing paradigm. Journal of Experimental Psychology:Human Perception and Performance, 30(4), 708-719, doi:10.1037/0096-1523.30.4.708.
Wenger, M. J., & Townsend, J. T. (2000). Basic response time tools for studying general processing capacity in attention, perception, and cognition. The Journal of General Psychology, 127(1), 67-99, doi:10.1080/00221300009598571.
Wickelgren, W. A. (1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41(1), 67-85, doi:10.1016/0001-6918(77)90012-9.

[1]	Dai Jinghua, Jia Ning. Accuracy of Judgments of Learning for The Primary School Students with Different Recall Tasks [J]. Studies of Psychology and Behavior, 2017, 15(2): 223-226.
[2]	Jiang Guangrong,Hu Bo. THE DEVELOPMENT OF MENTAL HEALTH-RELATED SELF SCALE [J]. Studies of Psychology and Behavior, 2006, 4(2): 95-100.