Studies of Psychology and Behavior

Studies of Psychology and Behavior >

2023 , Vol. 21 >Issue 6: 760 - 769

DOI: https://doi.org/10.12139/j.1672-0628.2023.06.006

?

Research on Test Assembly of Item Discrimination Index Under Polytomous Attributes and Mixed Scoring Items

  • Dafu MA ,
  • Chunying QIN ,
  • Xiaofeng YU ,
  • Cui HE
Expand
  • 1. School of Psychology, Jiangxi Normal University, Nanchang 330022
    2. Jinan Institute for Education and Teaching Research, Jinan 250002
    3. School of Mathematics and Information Science, Nanchang Normal University, Nanchang 330032

Received date: 2022-08-13

  Online published: 2024-01-24

Copyright

, 2023, Copyright reserved © 2023.
Fold

Abstract

In this study, two item discrimination indices (CDI, ADI) and two general modified item discrimination indices (GMCDI, GMADI) were used for polytomous attribute diagnosis test assembly. Two simulation studies were conducted considering three real-world testing scenarios: Study 1, an independent test of dichotomous and polytomous scoring items; Study 2, a mixed test of dichotomous and polytomous scoring items. The results indicate that GMCDI and GMADI can obtain the highest diagnosis accuracy, and are not affected by attribute hierarchy and test type.

Key words： item discrimination index; CD-ATA; polytomous attributes; dichotomous scoring; polytomous scoring

Cite this article

Dafu MA , Chunying QIN , Xiaofeng YU , Cui HE . Research on Test Assembly of Item Discrimination Index Under Polytomous Attributes and Mixed Scoring Items[J]. Studies of Psychology and Behavior, 2023 , 21(6) : 760 -769 . DOI: 10.12139/j.1672-0628.2023.06.006

References

蔡艳, 涂冬波. 属性多级化的认知诊断模型拓展及其Q矩阵设计. 心理学报, 2015, 47 (10): 1300- 1308.
  丁树良, 汪文义, 罗芬, 熊建华. (2016). 可达阵功能的不可替代性. 江西师范大学学报(自然科学版), 40(3), 290–294, 298.
  丁树良, 杨淑群, 汪文义. 可达矩阵在认知诊断测验编制中的重要作用. 江西师范大学学报(自然科学版), 2010, 34 (5): 490- 494.
  唐小娟, 丁树良, 毛萌萌, 俞宗火. 基于属性层级结构的认知诊断测验的组卷. 心理学探新, 2013, 33 (3): 252- 259.
  涂冬波, 蔡艳, 戴海琦, 丁树良. 一种多级评分的认知诊断模型: P-DINA模型的开发. 心理学报, 2010, 42 (10): 1011- 1020.
  汪文义, 宋丽红, 丁树良. 分类视角下认知诊断测验项目区分度指标及应用. 心理科学, 2018, 41 (2): 475- 483.
  王立君, 赵少勇, 昌维, 唐芳, 詹沛达. (2022). 重参数化多分属性DINA模型的多级评分拓广——基于等级反应模型. 心理科学, 45(1), 195–203.
  詹沛达, 边玉芳, 王立君. 重参数化的多分属性诊断分类模型及其判准率影响因素. 心理学报, 2016, 48 (3): 318- 330.
  Becker, B., Debeer, D., Sachse, K. A., & Weirich, S.. Automated test assembly in R: The eatATA package. Psych, 2021, 3 (2): 96- 112.
  Chang, H. H., & Ying, Z. L.. A global information approach to computerized adaptive testing. Applied Psychological Measurement, 1996, 20 (3): 213- 229.
  Chen, J. S., & de la Torre, J.. A general cognitive diagnosis model for expert-defined polytomous attributes. Applied Psychological Measurement, 2013, 37 (6): 419- 437.
  Finkelman, M., Kim, W., & Roussos, L. A.. Automated test assembly for cognitive diagnosis models using a genetic algorithm. Journal of Educational Measurement, 2009, 46 (3): 273- 292.
  Finkelman, M. D., de la Torre, J., & Karp, J. A.. Cognitive diagnosis models and automated test assembly: An approach incorporating response times. International Journal of Testing, 2020, 20 (4): 299- 320.
  Finkelman, M. D., Kim, W., Roussos, L., & Verschoor, A.. A binary programming approach to automated test assembly for cognitive diagnosis models. Applied Psychological Measurement, 2010, 34 (5): 310- 326.
  Henson, R., & Douglas, J.. Test construction for cognitive diagnosis. Applied Psychological Measurement, 2005, 29 (4): 262- 277.
  Henson, R., Roussos, L., Douglas, J., & He, X. M.. Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 2008, 32 (4): 275- 288.
  Huebner, A., & Wang, C.. A note on comparing examinee classification methods for cognitive diagnosis models. Educational and Psychological Measurement, 2011, 71 (2): 407- 419.
  Kuo, B. C., Pai, H. S., & de la Torre, J.. Modified cognitive diagnostic index and modified attribute-level discrimination index for test construction. Applied Psychological Measurement, 2016, 40 (5): 315- 330.
  Leighton, J. P., & Gierl, M. J.. Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practice, 2007, 26 (2): 3- 16.
  Lin, Y., Gong, Y. J., & Zhang, J.. An adaptive ant colony optimization algorithm for constructing cognitive diagnosis tests. Applied Soft Computing, 2017, 52, 1- 13.
  Lin, Y., Jiang, Y. S., Gong, Y. J., Zhan, Z. H., & Zhang, J.. A discrete multiobjective particle swarm optimizer for automated assembly of parallel cognitive diagnosis tests. IEEE Transactions on Cybernetics, 2019, 49 (7): 2792- 2805.
  Mullis, I. V. S., Martin, M. O., Ruddock, G. J., O’Sullivan, C. Y., Christine, Y., Arora, A., & Erberber, E. (2005). Timss 2007 assessment frameworks. Boston: International Association for the Evaluation of Educational Achievement.
  Rupp, A. A., Templin, J., & Henson, R. J. (2010). Diagnostic measurement: Theory, methods, and applications. New York: The Guilford Press.
  von Davier, M.. A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 2008, 61 (2): 287- 307.
  von Davier, M., & Lee, Y. S. (2019). Handbook diagnostic classification models. New York: Springer.
  Wang, S. Y., Zheng, Y., Zheng, C. J., Su, Y. H., & Li, P. Z.. An automated test assembly design for a large-scale Chinese proficiency test. Applied Psychological Measurement, 2016, 40 (3): 233- 237.
  Zeng, L. Y., Ding, S. L., & Gan, D. W. (2010). Test construction for cognitive diagnosis. In 2010 Asia-Pacific conference on wearable computing systems (pp. 12–15). Shenzhen: IEEE.
  Zhan, P. D., Wang, W. C., & Wang, L. (2017). A higher-order cognitive diagnosis model for polytomous attributes and polytomous responses. Poster presented at the annual meeting of the National Council on Measurement in Education, San Antonio.
Options
Outlines

/

〈 〉

Copyright © Editorial office of Studies of Psychology and Behavior
Tel: 022-23540231, 23541213　E-mail: psybeh@126.com