基于贝叶斯网模型的多级计分诊断测验分类及比较研究

doi:10.12139/j.1672-0628.2023.01.008

摘要/Abstract

摘要：

贝叶斯网模型提供了一种方便和直观的框架结构来表示变量间的关系，非常适合在诊断测验中对教育评估的内容进行建模。本研究将两种贝叶斯网分类模型与序列多级计分诊断模型S-GDINA进行综合比较。考察两种贝叶斯网分类模型与S-GDINA在Q矩阵正确界定和包含一定比例(25%、30%)的错误时，两者对被试的分类性能；并将贝叶斯网分类模型应用到实证数据中，展示贝叶斯网分类模型在实证数据中的分类过程和分类性能。研究结果表明：当Q矩阵由专家正确界定时，朴素贝叶斯分类模型的分类效果与S-GDINA模型相差不大，同样可以达到很好的分类效果，树增广的朴素贝叶斯分类模型的分类性能也能达到良好。实证结果进一步表明，将贝叶斯网分类模型应用于教育测量领域中的诊断分类工具是有其优势和可行的，尤其是当测验数据对于所选用诊断模型的拟合较差、测验的Q矩阵中包含错误或测验数据中包含较多的噪音时。

关键词: 认知诊断, 序列多级计分诊断模型, 贝叶斯网络

Abstract:

In recent years, Bayesian networks have been widely used in the field of artificial intelligence, but have received relatively little attention in the field of psychology. Most of the existing studies have applied Bayesian networks to binary items. This research combines Bayesian networks with polytomous items, which has important theoretical and practical implications. Two Bayesian network classification models were comprehensively compared with the sequential-GDINA (S-GDINA). The results showed that the classification performance of the naive Bayesian classifier (NBC) was comparable to that of the S-GDINA model, which could achieve equally good classification performance. Also, the classification performance of the tree augmented naive Bayesian classifier (TAN) could achieve good classification performance when the percentage of errors contained in the Q matrix was 0% (the Q matrix was correctly defined by the expert). Both the NBC and the TAN had better classification results in the polytomous scored test. The response classification consistency of the NBC and TAN reached 84% and 71%, respectively, indicating that Bayesian networks can be well applied to polytomous scored items.

Key words: cognitive diagnosis, S-GDINA, Bayesian networks

中图分类号:

B842

喻晓锋, 肖遇春, 秦春影. 基于贝叶斯网模型的多级计分诊断测验分类及比较研究[J]. 心理与行为研究, 2023, 21(1): 49-57.

Xiaofeng YU, Yuchun XIAO, Chunying QIN. A Comprehensive Comparison of Bayesian Networks Classification Model and Sequential Polytomous Scoring Diagnosis Model[J]. Studies of Psychology and Behavior, 2023, 21(1): 49-57.

图/表 11

参考文献

	马奕帆. (2021). 基于三种结构学习算法的属性层级结构估计研究与应用(硕士学位论文). 江西师范大学, 南昌.
	肖遇春. (2021). 基于贝叶斯网模型的多级计分诊断测验分类研究(硕士学位论文). 江西师范大学, 南昌.
	喻晓锋, 丁树良, 秦春影, 陆云娜贝叶斯网在认知诊断属性层级结构确定中的应用. 心理学报, 2011, 43 (3): 338- 346.
	喻晓锋, 罗照盛, 高椿雷, 秦春影 Q矩阵包含错误的诊断测验分类准确性比较. 心理科学, 2014, 37 (6): 1478- 1484. DOI
	Almond, R. G., DiBello, L. V., Moulder, B., & Zapata-Rivera, J. Modeling diagnostic assessments with Bayesian networks. Journal of Educational Measurement, 2007, 44 (4): 341- 359. DOI
	Béland, A., & Mislevy, R. J. Probability-based inference in a domain of proportional reasoning tasks. Journal of Educational Measurement, 1996, 33 (1): 3- 27. DOI
	Chen, P., Xin, T., Wang, C., & Chang, H. H. Online calibration methods for the DINA model with independent attributes in CD-CAT. Psychometrika, 2012, 77 (2): 201- 222. DOI
	de la Torre, J. The generalized DINA model framework. Psychometrika, 2011, 76 (2): 179- 199. DOI
	Friedman, N., Geiger, D., & Goldszmidt, M. Bayesian network classifiers. Machine Learning, 1997, 29 (2): 131- 163.
	Gitomer, D. H., Steinberg, L. S., & Mislevy, R. J. (1995). Diagnostic assessment of troubleshooting skill in an intelligent tutoring system. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 73–101). Hillsdale, NJ: Erlbaum.
	Lee, J. (2003). Diagnosis of bugs in multi-column subtraction using Bayesian networks (Unpublished doctorial dissertation). Columbia University, New York.
	Lee, J. Y., & Corter, J. E. Diagnosis of subtraction bugs using Bayesian networks. Applied Psychological Measurement, 2011, 35 (1): 27- 47. DOI
	Lee, Y. S., Park, Y. S., & Taylan, D. A cognitive diagnostic modeling of attribute mastery in Massachusetts, Minnesota, and the U.S. national sample using the TIMSS 2007. International Journal of Testing, 2011, 11 (2): 144- 177. DOI
	Leighton, J. P., & Gierl, M. J. (2007). Cognitive diagnostic assessment for education. New York: Cambridge University Press.
	Ma, W. C., & de la Torre, J. A sequential cognitive diagnosis model for polytomous responses. British Journal of Mathematical and Statistical Psychology, 2016, 69 (3): 253- 275. DOI
	Ma, W. C., & de la Torre, J. GDINA: An R package for cognitive diagnosis modeling. Journal of Statistical Software, 2020, 93 (14): 1- 26.
	Mihaljević, B., Bielza, C., Larrañaga, P. bnclassify: Learning Bayesian network classifiers. The R Journal, 2018, 10 (2): 455- 468.
	Mislevy, R. J. (1995). Probability-based inference in cognitive diagnosis. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds), Cognitively diagnostic assessment (pp. 43–71). Hillsdale, NJ: Erlbaum.
	Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Almond, R. G., & Johnson, L. Making sense of data from complex assessments. Applied Measurement in Education, 2002, 15 (4): 363- 389. DOI
	Neapolitan, R. E. (2003). Learning Bayesian networks. Harlow, United Kingdom: Prentice Hall.
	Nichols, P. D. A framework for developing cognitively diagnostic assessments. Review of Educational Research, 1994, 64 (4): 575- 603. DOI
	Pearl, J. (1985). Bayesian networks: A model of self-activated memory for evidential reasoning. In Proceedings of the 7th Conference of the Cognitive Science Society (pp. 329–334). Irvine, CA.
	Rupp, A. A., & Templin, J. L. Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research and Perspective, 2008, 6 (4): 219- 262. DOI
	Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores Psychometrika, 34 (1), 1–97.
	Vomlel, J. (2004). Bayesian networks in educational testing. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 12(S1), 83–100.
	von Davier, M., & Lee, Y. S. (2019). Handbook of diagnostic classification models. New York: Springer.
	Wang, L. L., Xin, T., & Liu, Y. L. An improved parameter-estimating method in Bayesian networks applied for cognitive diagnosis assessment. Frontiers in Psychology, 2021, 12, 665441. DOI
	Yadav, S., & Shukla, S. (2016). Analysis of k-fold cross-validation over hold-out validation on colossal datasets for quality classification. In 2016 IEEE 6th International Conference on Advanced Computing (pp. 78–83). Bhimavaram, India.
	Yu, X. F., & Cheng, Y. Data-driven Q-matrix validation using a residual-based statistic in cognitive diagnostic assessment. British Journal of Mathematical and Statistical Psychology, 2020, 73 (S1): 145- 179. DOI

n	猜测/失误参数(%)	Q矩阵错误比例r(%)	PCCR	ACCR
1000	10	0	0.942	0.988
		25	0.895	0.978
		30	0.881	0.975
	15	0	0.853	0.969
		25	0.762	0.947
		30	0.716	0.936
2000	10	0	0.944	0.988
		25	0.897	0.978
		30	0.883	0.975
	15	0	0.856	0.969
		25	0.767	0.948
		30	0.721	0.937
4000	10	0	0.944	0.988
		25	0.898	0.979
		30	0.887	0.976
	15	0	0.856	0.969
		25	0.777	0.951
		30	0.735	0.940

n	猜测/失误参数(%)	Q矩阵错误比例r(%)	PCCR	ACCR
1000	10	0	0.942	0.988
		25	0.895	0.978
		30	0.881	0.975
	15	0	0.853	0.969
		25	0.762	0.947
		30	0.716	0.936
2000	10	0	0.944	0.988
		25	0.897	0.978
		30	0.883	0.975
	15	0	0.856	0.969
		25	0.767	0.948
		30	0.721	0.937
4000	10	0	0.944	0.988
		25	0.898	0.979
		30	0.887	0.976
	15	0	0.856	0.969
		25	0.777	0.951
		30	0.735	0.940

n	贝叶斯网分类模型	错误比例r(%)	分类效果指标	猜测/失误参数
n	贝叶斯网分类模型	错误比例r(%)	分类效果指标	10%	15%
1000	NBC	0	PCCR	0.921	0.808
		0	ACCR	0.984	0.958
		25	PCCR	0.889	0.761
		25	ACCR	0.977	0.946
		30	PCCR	0.891	0.766
		30	ACCR	0.977	0.948
	TAN	0	PCCR	0.786	0.615
		0	ACCR	0.952	0.905
		25	PCCR	0.738	0.568
		25	ACCR	0.940	0.890
		30	PCCR	0.743	0.571
		30	ACCR	0.941	0.891
2000	NBC	0	PCCR	0.931	0.829
		0	ACCR	0.986	0.963
		25	PCCR	0.904	0.790
		25	ACCR	0.980	0.953
		30	PCCR	0.903	0.789
		30	ACCR	0.979	0.953
	TAN	0	PCCR	0.838	0.683
		0	ACCR	0.965	0.925
		25	PCCR	0.797	0.646
		25	ACCR	0.955	0.915
		30	PCCR	0.796	0.644
		30	ACCR	0.954	0.914
4000	NBC	0	PCCR	0.938	0.841
		0	ACCR	0.987	0.966
		25	PCCR	0.913	0.806
		25	ACCR	0.982	0.957
		30	PCCR	0.910	0.801
		30	ACCR	0.981	0.956
	TAN	0	PCCR	0.867	0.742
		0	ACCR	0.971	0.942
		25	PCCR	0.836	0.710
		25	ACCR	0.964	0.933
		30	PCCR	0.832	0.705
		30	ACCR	0.963	0.931

n	贝叶斯网分类模型	错误比例r(%)	分类效果指标	猜测/失误参数
n	贝叶斯网分类模型	错误比例r(%)	分类效果指标	10%	15%
1000	NBC	0	PCCR	0.921	0.808
		0	ACCR	0.984	0.958
		25	PCCR	0.889	0.761
		25	ACCR	0.977	0.946
		30	PCCR	0.891	0.766
		30	ACCR	0.977	0.948
	TAN	0	PCCR	0.786	0.615
		0	ACCR	0.952	0.905
		25	PCCR	0.738	0.568
		25	ACCR	0.940	0.890
		30	PCCR	0.743	0.571
		30	ACCR	0.941	0.891
2000	NBC	0	PCCR	0.931	0.829
		0	ACCR	0.986	0.963
		25	PCCR	0.904	0.790
		25	ACCR	0.980	0.953
		30	PCCR	0.903	0.789
		30	ACCR	0.979	0.953
	TAN	0	PCCR	0.838	0.683
		0	ACCR	0.965	0.925
		25	PCCR	0.797	0.646
		25	ACCR	0.955	0.915
		30	PCCR	0.796	0.644
		30	ACCR	0.954	0.914
4000	NBC	0	PCCR	0.938	0.841
		0	ACCR	0.987	0.966
		25	PCCR	0.913	0.806
		25	ACCR	0.982	0.957
		30	PCCR	0.910	0.801
		30	ACCR	0.981	0.956
	TAN	0	PCCR	0.867	0.742
		0	ACCR	0.971	0.942
		25	PCCR	0.836	0.710
		25	ACCR	0.964	0.933
		30	PCCR	0.832	0.705
		30	ACCR	0.963	0.931

属性掌握模式	被试人数	属性掌握模式	被试人数
2	91	178	31
57	104	206	27
107	115	227	240
143	26	256	189