The Evaluation of the Quality of Postgraduate Entrance Examination Based on IRT

doi:10.12139/j.1672-0628.2023.02.015

Abstract

Abstract: The quality evaluation of postgraduate entrance examination is important in management. Based on the answering data of 22,126 subjects in the psychology major of the postgraduate entrance examination in 2022, the quality of the test paper was analyzed by using the IRT through SPSS 21.0 and R. The results showed that, 1) the structure of the test paper was completely consistent with the requirements of the test outline; 2) the range of subjects’ abilities was widely distributed; 3) the test information basically met the requirements. The difficulty level of the test paper is medium to easy; and there are still some poor items, and the proportion of average items is also relatively high. From the perspective of information function, 1) the maximum information content of most items was higher than the expected information content; 2) the proportion of the maximum information content provided by Experimental Psychology, Psychological Statistics and Measurement was higher than the proportion of the information content that should be provided; 3) the test information function curve was on the left as a whole, which was not good for differentiation of groups of subjects with higher ability.

Key words: item response theory (IRT), postgraduate entrance examination, evaluation

摘要： 研究生招生考试命题质量评价是考试管理的重要环节。针对2022年全国硕士研究生招生考试心理学专业基础科目，随机抽取22,126份作答数据，利用SPSS21.0和R软件，采用双参数Logistic模型和拓广分部评分模型对命题质量进行评价。结果显示：试卷结构与考试大纲的要求完全一致；各种题型对知识点的考查均偏重基础；考生能力范围分布较广；测验信息量基本满足要求。从难度来看，试卷难度中等偏易；从区分度来看，仍有部分差级试题，中级试题占比也偏高。从信息函数来看，绝大多数试题的最大信息量都高于期望信息量；实验心理学、心理统计与测量实际提供的最大信息量比例高于应提供的信息量比例；测验信息函数曲线整体偏左，对于能力水平较高的考生群体区分度有待提高。

关键词: 项目反应理论, 研究生招生考试, 评价

CLC Number:

B849

SONG Xueling, LIANG Zhengyan. The Evaluation of the Quality of Postgraduate Entrance Examination Based on IRT[J]. Studies of Psychology and Behavior, 2023, 21(2): 253-259.

宋学玲, 梁正妍. 基于项目反应理论的研究生招生考试命题质量评价[J]. 心理与行为研究, 2023, 21(2): 253-259.

References

胡姗. (2015). 基于GPCM和3PLM的CAT研究(硕士学位论文). 江西师范大学, 南昌.
纪凌开. (2002). 拓广分部评分模型(GPCM)及应用EM算法的参数估计研究(硕士学位论文). 江西师范大学, 南昌.
罗照盛. (2012). 项目反应理论基础. 北京: 北京师范大学出版社.
罗照盛, 欧阳雪莲, 漆书青, 戴海琦, 丁树良. (2008). 项目反应理论等级反应模型项目信息量. 心理学报, 40(11), 1212–1220
漆书青, 戴海崎, 丁树良. (1998). 现代教育与心理测量学原理. 南昌: 江西教育出版社.
沈励, 万雅奇. (2022). 高中学业水平等级性考试数据分析拓展研究. 中国考试, (5), 54–63
涂冬波, 蔡艳, 戴海琦, 丁树良. (2011). 多维项目反应理论: 参数估计及其在心理测验中的应用. 心理学报, 43(11), 1329–1340
熊建华, 丁树良, 漆书青, 戴海崎. (2002). 用测验信息量分析试卷质量. 江西师范大学学报(自然科学版), 26(3), 225–228
闫成海, 杜文久, 宋乃庆, 张健. (2014). 高考数学中考试评价的研究——基于CTT与IRT的实证比较. 华东师范大学学报(教育科学版), 32(03), 10–18
闫培香. (2021). 项目反应理论在英语考试分数可比性研究中的应用. 教育与考试, (4), 53–57
赵守盈, 何妃霞, 陈维, 罗杰, 关丹丹. (2012). Rasch模型在研究生入学考试质量分析中的应用. 教育研究, 33(6), 61–65
赵守盈, 石艳梅, 朱丹. (2013). 项目反应理论在大规模选拔性考试试题质量评价中的应用. 教育学报, 9(1), 71–77
庄然, 郑淑园, 田甜, 王振禹, 肖天, 刘侃. (2020). 项目反应理论在基础医学综合测试免疫学试题中的应用. 细胞与分子免疫学杂志, 36(1), 86–94
Gonçalves, F. B., Venturelli, S. L. J., & Loschi, R. H. (2023). Flexible Bayesian modelling in dichotomous item response theory using mixtures of skewed item curves. British Journal of Mathematical and Statistical Psychology, 76(1), 69–86.
Lahner, F. M., Schauber, S., Lörwald, A. C., Kropf, R., Guttormsen, S., Fischer, M. R., & Huwendiek, S. (2020). Measurement precision at the cut score in medical multiple choice exams: Theory matters. Perspectives on Medical Education, 9(4), 220–228.
Lee, S. S., & von Davier, M. (2020). Improving measurement properties of the PISA home possessions scale through partial invariance modeling. Psychological Test and Assessment Modeling, 62(1), 55–83.
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–176.
Rahim, A., & Haryanto, H. (2021). Implementation of item response theory (IRT) Rasch model in quality analysis of final exam tests in mathematics. Journal of Research and Educational Research Evaluation, 10(2), 57–65.
Strachan, T., Cho, U. H., Ackerman, T., Chen, S. H., de la Torre, J., & Ip, E. H. (2022). Evaluation of the linear composite conjecture for unidimensional IRT scale for multidimensional responses. Applied Psychological Measurement, 46(5), 347–360.
Young, J. W., Morgan, R., Rybinski, P., Steinberg, J., & Wang, Y. (2013). Assessing the test information function and differential item functioning for the toefl junior standard test. ETS Research Report Series, 2013(1), i–27.

[1]	Jiao CHEN, Meng XIONG. Effects of Relative Deprivation on College Students’ Social Anxiety: A Moderated Mediating Model [J]. Studies of Psychology and Behavior, 2023, 21(1): 65-71.
[2]	HUANG Jie, ZHU Dan, ZHOU Limin, LIU Wang. Intergenerational Transmission of Core Self-Evaluations: Chain Mediation of Parental Support and Offspring’s Basic Psychological Need [J]. Studies of Psychology and Behavior, 2022, 20(4): 515-521.
[3]	ZHU Zhaohong, SUN Lingling. Automatic Evaluation of Supraliminal and Subliminally Presentation of Exercis-Related Stimuli in Different Exercise Groups [J]. Studies of Psychology and Behavior, 2022, 20(2): 182-189.
[4]	TIAN Hongjie. Effect of High Standards of Perfectionism on College Students’ Procrastination: Chain Mediation Effect Analysis [J]. Studies of Psychology and Behavior, 2019, 17(5): 668-674.
[5]	LUO Jie, CHEN Wei, YANG Guifang, WU Danping, LIU Tuo. The Effect of Proactive Personality on College Students’ Procrastination: The Mediating Roles of Core Self-evaluations [J]. Studies of Psychology and Behavior, 2019, 17(5): 692-698.
[6]	QI Bing, YANG Lizhu, YAO Yibei. Effects of Emotion Regulation Strategies on Memory of Junior High School Students' Educational Materials Induced by Different Emotions [J]. Studies of Psychology and Behavior, 2019, 17(2): 202-208.
[7]	WANG Daoyang, WEI Wei, YIN Xin. Migration Children's Inferiority and Perceived Academic Self-efficacy: The Moderating Role of Emotion Regulation Strategies [J]. Studies of Psychology and Behavior, 2019, 17(1): 48-55.
[8]	LIANG Limei, LI Jiagen, JIA Jiangxiao, SONG Haixiu, LEI Yi, LI Hong, ZHANG Wenhai. Different Evaluations on Inhibition Function in Trait Anxiety:An ERP Study [J]. Studies of Psychology and Behavior, 2018, 16(3): 335-341.
[9]	HU Xinyi, CHEN Yinghe. The Effect of Incremental Belief about Intelligence on Self Evaluation in High Threat Condition: the Influence of Order and Self-Esteem [J]. Studies of Psychology and Behavior, 2018, 16(2): 195-201.
[10]	LUO Dasen. A Crude Evaluation on the Two Roots of the Reproducibility Crisis in Psychology [J]. Studies of Psychology and Behavior, 2017, 15(5): 577-586.
[11]	WANG Hanlin, MO Lei. “Action-Evaluation” Effect: The Interaction Between Body Movement and Cognitive Activity [J]. Studies of Psychology and Behavior, 2017, 15(5): 626-632.
[12]	GUO Jichang, YE Yiduo. How Core Self-Evaluations Influences Life Satisfaction in Adolescents: The Mediating Effects of Hope [J]. Studies of Psychology and Behavior, 2017, 15(5): 663-668.
[13]	Zhang Min,Zhang Ping,Lu Jiamei. The Effects of Cognitive Evaluation on the Generation of Gratitude Emotion [J]. Studies of Psychology and Behavior, 2015, 13(3): 380-384.
[14]	Yang Chun, Lu Haidong. Different Evaluative Feedback Sign Effect on College Students'Learning Motivation and Achievement [J]. Studies of Psychology and Behavior, 2015, 13(2): 237-241.
[15]	Lan Xiaofang,Zhang Jinkun. The Gender Difference of FNE: Based on The Comparison of Implicit And Explicit Measurement [J]. Studies of Psychology and Behavior, 2015, 13(1): 25-30.