QualityAnalysis in National Teacher Certification Examination Based on Multivariate Generalizability Theory

Abstract

Abstract: National Teacher Certification Examination is one of the reform program identified in The National Medium and Long-term Education Reform and Development Plan Outline (2010-2020).In this article, we adopted multivariate generalizability theory (MGT) to appraise test reliability, combination of scores, test structure optimization based on data of National Teacher Certification Examination. The result showed that: a) the reliability coefficient of whole test is 0.707, the result is acceptably high; b) the ratio of each subtest's contribution to total variance is inaccordance with test-setters' anticipation, in addition to writing ability, the remaining part of the error is less than 7%; c) to promote test reliability, it is recommended to reduce the number of objective questions and increase the number of subjective question.

Key words: multivariate generalizability theory, national teacher certification examination, reliability

摘要： 教师资格考试是国家教育规划纲要确定的改革项目之一。本文运用多元概化理论对中小学教师资格考试笔试科目《综合素质》（中学）试卷的测量信度、试卷总分合成、试卷结构优化等问题进行探讨。研究发现：本次测试的测量信度较高，为0.707；各测量分项对全域总分的方差贡献比例与试卷赋分意图基本一致，除写作能力外，其余部分误差在7%以内；可以采取减少客观题数量，增加主观题数量的方式来提高信度。

关键词: 多元概化理论, 中小学教师资格考试, 信度

CLC Number:

G443

YANG Hongbo, ZHAO Xuan. QualityAnalysis in National Teacher Certification Examination Based on Multivariate Generalizability Theory[J]. Studies of Psychology and Behavior, 2019, 17(2): 253-258.

杨宏博, 赵轩. 基于多元概化理论的中小学教师资格考试质量分析——以《综合素质》（中学）科目为例[J]. 心理与行为研究, 2019, 17(2): 253-258.

References

安哲锋, 骆方, 张厚粲. (2008). 多元概化理论在评定量表编制中的作用——以音像教材测评数据分析为例. 心理科学, 31(5), 1192-1194
白娟. (2014). 基于多元概化理论的全国硕士研究生中医综合考试质量分析. 天津中医药大学学报, 33(1), 44-46
陈维, 何壮, 赵守盈. (2016). 高考语文模考试卷题型结构的多元概化分析. 教学与管理, (18), 116-118
陈维, 卢聪, 杨晓晓, 张进辅. (2016). 领悟社会支持量表的多元概化分析. 心理学探新, 36(1), 75-78
陈维, 赵守盈. (2016). 大学生网络成瘾量表(修订版)的多元概化分析. 心理与行为研究, 14(1), 95-99, doi:10.3969/j.issn.1672-0628.2016.01.014
关丹丹, 任子朝. (2009). 应用概化理论评价课标后高考数学试卷. 数学通报, 48(11), 18-24, doi:10.3969/j.issn.0583-1458.2009.11.006
教育部. (2013). 中小学教师资格考试暂行办法. 2017-06-25取自http://www.moe.gov.cn/srcsite/A10/s7151/201308/t20130821_156643.html
教育部考试中心[C1]. (2017). 中小学教师资格考试统计分析报告(内部材料).
黎光明, 张敏强. (2017). 高校教师教学水平评价多元概化理论权重效应分析. 心理发展与教育, 33(1), 122-128
刘远我, 张厚粲. (1998). 概化理论在作文评分中的应用研究. 心理学报, 30(2), 211-218
罗杰, 戴晓阳. (2016). 中文形容词大五人格量表的初步编制Ⅲ:基于概化理论的视角. 中国临床心理学杂志, 24(1), 88-94
罗照盛, 郭小军. (2014). 认知行为实验研究中最佳素材容量的选择与确定:多元概化理论应用. 心理学报, 46(6), 876-884
田霖, 韦小满, 王桥影, 赵晓茫. (2013). 基于概化理论构建自学考试信度分析框架. 考试研究, (5), 27-33
杨志明, 张雷. (2003). 测评的概化理论及其应用. 北京:教育科学出版社.
Brennan, R. L. (2001a). Generalizability theory. New York:Sprin-ger-Verlag.
Brennan, R. L. (2001b). An essay on the history and future of reliability from the perspective of replications. Journal of Educational Measurement, 38(4), 295-317, doi:10.1111/jedm.2001.38.issue-4.
Brennan, R. L. (2001c). Manual for mGENOVA. Iowa City, IA:Iowa Testing Programs, University of Iowa.

[1]	MIAO Xiuying, CHI Lizhong. The Impact of the Affect Heuristic on Risky Choice [J]. Studies of Psychology and Behavior, 2022, 20(3): 325-331.
[2]	WANG Yanli, CHEN Hong. The Development of Sense of Gain Questionnaire Among Urban Community Residents [J]. Studies of Psychology and Behavior, 2021, 19(5): 665-670.
[3]	LIU Wen, MENG Xiangrui, CHE Hanbo, ZHANG Jiaqi, SHEN Yi. Development, Psychometric Reliability and Validity of Pupils’ Creative Personality Questionnaire Based on Teacher’s Descriptions [J]. Studies of Psychology and Behavior, 2020, 18(5): 624-630.
[4]	WANG Zhanqi, ZHANG Xingli. The Validity and Reliability of the Chinese Version of the Creative Achievement Questionnaire [J]. Studies of Psychology and Behavior, 2020, 18(3): 390-397.
[5]	ZHANG Lina, LU Haidong. Development of Chinese High School Students Perceived Classroom Teaching Approaches Scale [J]. Studies of Psychology and Behavior, 2019, 17(6): 773-779.
[6]	HAO Jiajia, CHEN Yinghe, LIU Tuo, BAI Xuejun. The Development of General Thinking Test for Children [J]. Studies of Psychology and Behavior, 2019, 17(5): 620-626.
[7]	DAI Binrong. The Development of the Rural Children's Social Adaptation Scale [J]. Studies of Psychology and Behavior, 2019, 17(4): 504-511.
[8]	MA Huixia, LIU Huifang, ZHAI Yongfang, LI Guifang. Revision and Verification of the Test of Self-Conscious Affect for Adolescents [J]. Studies of Psychology and Behavior, 2019, 17(2): 240-245.
[9]	GUO Jing, WANG Ying, JI Lili, JIA Liping, LU Guohua. Development and Validation of Psychological Crisis Vulnerability Scale [J]. Studies of Psychology and Behavior, 2018, 16(6): 847-853.
[10]	LUO Jie, ZHOU Yuan, CHEN Wei, PAN Yun, ZHAO Shouying. A Multivariate Generalizability Analysis of the Adolescent Student Burnout Inventory [J]. Studies of Psychology and Behavior, 2018, 16(5): 665-669,712.
[11]	CHEN Qiu-yan, ZHANG Chen-guang, CHENG Ke. Revision of Posttraumatic Growth Inventory Among Teachers in Wen-chuan Earthquake [J]. Studies of Psychology and Behavior, 2018, 16(5): 670-677.
[12]	FANG Xiaoyi, YUAN Xiaojiao, HU Wei, DENG Linyuan, LIN Xiuyun. The Development of College Students Mental Health Screening Scale [J]. Studies of Psychology and Behavior, 2018, 16(1): 111-118.
[13]	WANG Hua, JIAO Runkai. The Establishment and Examination of Reliability and Validity of Career Management Competencies Questionnaire in College Students [J]. Studies of Psychology and Behavior, 2017, 15(6): 793-798.
[14]	Qi Yajing, Wu Xinchun. The Development of Job Crafting Questionnaire of Primary and Secondary School Teachers [J]. Studies of Psychology and Behavior, 2016, 14(4): 501-506.
[15]	Chen Wei, Zhao Shouying. A Multivariate Generalizability Analysis of the Chinese Internet Addiction Scale(Revision) [J]. Studies of Psychology and Behavior, 2016, 14(1): 95-99.