人工智能技术在青少年未来取向评分中的适用性

doi:10.12139/j.1672-0628.2026.03.014

摘要/Abstract

摘要：

作品分析法能够有效揭示出青少年未来取向的发展趋势，但由于评分过程存在高成本、评分者素质高要求和主观性的问题，作品分析法在未来取向测量中难以广泛使用。为了解决以上问题，本研究探索基于人工智能技术的作品分析法在青少年未来取向评分中的可行性。选取307名来自不同教育阶段学生的未来取向作品，以人工评分为基准，考察智能评分结果的信度、区分度、重测信度以及效标关联效度。结果发现，智能评分信效度各指标良好，与人工评分一致性较高，初步验证了基于人工智能技术的作品分析法在未来取向评分中的可行性。研究表明，智能评分与人工评分极其接近并具有稳定性，有助于促进作品分析法在未来取向测量中的广泛应用。

关键词: 作品分析法, 人工智能技术, 未来取向

Abstract:

The work analysis method is effective for revealing developmental trends in adolescents’ future orientation, but its widespread application is hindered by high costs, stringent rater requirements, and subjectivity in the scoring process. To address these issues, this study explored the feasibility of an artificial intelligence (AI)-based work-analysis method in scoring adolescents’ future orientation. Collecting datasets of future orientation works from 307 students at various educational stages and using manual scoring as benchmark, we evaluated the reliability, discriminability, test-retest reliability, and criterion-related validity of AI scoring. The results showed that AI scoring demonstrated good psychometric properties on all indices and achieved high consistency with the manual ratings. The feasibility of AI-based work-analysis method in future oriented scoring is thus validated preliminarily. Furthermore, in view of its high consistency and stability, AI scoring can facilitate the broader application of the work-analysis method in future orientation measurement.

Key words: work analysis method, artificial intelligence technology, future orientation

中图分类号:

B849

盖笑松, 刘本扬, 邸楠. 人工智能技术在青少年未来取向评分中的适用性[J]. 心理与行为研究, 2026, 24(3): 393-400.

Xiaosong GAI, Benyang LIU, Nan DI. Applicability of Artificial Intelligence Technology in the Assessment of Adolescents’ Future Orientation[J]. Studies of Psychology and Behavior, 2026, 24(3): 393-400.

图/表 5

参考文献 0

	董艳, 夏亮亮, 李心怡, 侯彦华. ChatGPT赋能学生学习的路径探析. 电化教育研究, 2023, 44 (12): 14- 20, 34. DOI
	冯志伟, 张灯柯. 语言模型与人工智能. 外语研究, 2024, 41 (1): 1- 19, 112. DOI
	刘本扬, 盖笑松, 王国霞. 青少年未来取向三种测试途径的比较. 心理科学, 2022, 45 (5): 1152- 1158.
	刘明, 吴忠明, 廖剑, 任伊灵, 苏逸飞. 大语言模型的教育应用: 原理、现状与挑战——从轻量级BERT到对话式ChatGPT. 现代教育技术, 2023, 33 (8): 19- 28. DOI
	刘霞, 黄希庭, 普彬, 毕翠华. 未来取向研究概述. 心理科学进展, 2010, 18 (3): 385- 393. DOI
	陆俊花. 人工智能背景下机器评分与人工评分的效度比较——以英语学习者故事复述评分为例. 成都师范学院学报, 2022, 38 (3): 84- 92.
	吴虑, 杨磊. ChatGPT赋能学习何以可能. 电化教育研究, 2023, 44 (12): 28- 34. DOI
	肖国亮, 马磊, 袁峰, 郭成锋, 邢金宝. 智能评分技术应用效果的评价研究. 中国考试, 2023, (10): 17- 27.
	肖国亮, 马磊, 袁峰, 邢金宝. 人事考试与测评领域人工智能应用新探索. 中国人事科学, 2024, (10): 1- 10. DOI
	杨宗凯, 王俊, 吴砥, 陈旭. ChatGPT/生成式人工智能对教育的影响探析及应对策略. 华东师范大学学报(教育科学版), 2023, 41 (7): 26- 35.
	Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., … Dehghani, M. Perils and opportunities in using large language models in psychological research. PNAS Nexus, 2024, 3 (7): 245. DOI
	Barnett, M. D., Melugin, P. R., & Hernandez, J. Time perspective, intended academic engagement, and academic performance. Current Psychology, 2020, 39 (2): 761- 767. DOI
	Burns, E. C., Martin, A. J., & Collie, R. J. A future time perspective of secondary school students’ academic engagement and disengagement: A longitudinal investigation. Journal of School Psychology, 2021, 84, 109- 123. DOI
	Cao, X. B., & Kosinski, M. Large language models know how the personality of public figures is perceived by the general public. Scientific Reports, 2024, 14 (1): 6735. DOI
	Chen, H., Wu, Y. H., Jiang, L., Xu, B. F., Gao, X. P., & Cai, W. J. Future orientation and perceived employability of Chinese undergraduates: A moderated mediation model. Current Psychology, 2023, 42 (31): 27127- 27140. DOI
	Chen, P., & Vazsonyi, A. T. Future orientation, school contexts, and problem behaviors: A multilevel study. Journal of Youth and Adolescence, 2013, 42 (1): 67- 81. DOI
	Demszky, D., Yang, D. Y., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., ... Pennebaker, J. W. Using large language models in psychology. Nature Reviews Psychology, 2023, 2 (11): 688- 701.
	Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., ... Wright, R. “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 2023, 71, 102642. DOI
	Gilardi, F., Alizadeh, M., & Kubli, M. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences of the United States of America, 2023, 120 (30): e2305016120. DOI
	Iliev, R., Dehghani, M., & Sagi, E. Automated text analysis in psychology: Methods, applications, and future developments. Language and Cognition, 2015, 7 (2): 265- 290. DOI
	Macanovic, A., & Przepiorka, W. A systematic evaluation of text mining methods for short texts: Mapping individuals' internal states from online posts. Behavior Research Methods, 2024, 56 (4): 2782- 2803. DOI
	Markowitz, D. M. Can generative AI infer thinking style from language? Evaluating the utility of AI as a psychological text analysis tool. Behavior Research Methods, 2024, 56 (4): 3548- 3559. DOI
	OpenAI. (2023). ChatGPT (Mar 14 version) [Large language model]. Retrieved April 11, 2025, from https://chat.openai.com/chat
	Pangakis, N., Wolken, S., & Fasching, N. (2023). Automated annotation with generative AI requires validation. Retrieved April 11, 2025, from https://doi.org/10.48550/arXiv.2306.00176
	Peters, H., & Matz, S. C. Large language models can infer psychological dispositions of social media users. PNAS Nexus, 2024, 3 (6): 231. DOI
	Przepiorka, A., Blachnio, A., & Cudo, A. The role of depression, personality, and future time perspective in internet addiction in adolescents and emerging adults. Psychiatry Research, 2019, 272, 340- 348. DOI
	Rathje, S., Mirea, D. M., Sucholutsky, I., Marjieh, R., Robertson, C. E., & van Bavel, J. J. GPT is an effective tool for multilingual psychological text analysis. Proceedings of the National Academy of Sciences of the United States of America, 2023, 121 (34): e2308950121. DOI
	Reiss, M. V. (2023). Testing the reliability of ChatGPT for text annotation and classification: A cautionary remark. Retrieved April 11, 2025, from https://doi.org/10.48550/arXiv.2304.11085
	Stoddard, S. A., Zimmerman, M. A., & Bauermeister, J. A. Thinking about the future as a way to succeed in the present: A longitudinal study of future orientation and violent behaviors among African American youth. American Journal of Community Psychology, 2011, 48 (3−4): 238- 246. DOI
	Strachan, J. W. A., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., ... Becchio, C. Testing theory of mind in large language models and humans. Nature Human Behaviour, 2024, 8 (7): 1285- 1295. DOI
	Thorp, H. H. ChatGPT is fun, but not an author. Science, 2023, 379 (6630): 313. DOI
	Törnberg, P. (2023). ChatGPT-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. Retrieved April 11, 2025, from https://doi.org/10.48550/arXiv.2304.06588
	van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. ChatGPT: Five priorities for research. Nature, 2023, 614 (7947): 224- 226. DOI
	Walker, T. L., & Tracey, T. J. G. The role of future time perspective in career decision-making. Journal of Vocational Behavior, 2012, 81 (2): 150- 158. DOI
	Wills, T. A., Sandy, J. M., & Yaeger, A. M. Time perspective and early-onset substance use: A model based on stress-coping theory. Psychology of Addictive Behaviors, 2001, 15 (2): 118- 125. DOI

	小五(n=84)	初二(n=78)	高二(n=71)	大二(n=74)
人工评分	43.95±5.55	50.76±8.98	50.49±9.54	55.60±11.71
智能评分	42.48±7.01	52.42±9.35	51.02±8.64	55.00±10.10
Cohen’s d	0.23	−0.18	−0.06	0.05
组内相关系数	0.76**	0.80**	0.75**	0.78**

	小五(n=84)	初二(n=78)	高二(n=71)	大二(n=74)
人工评分	43.95±5.55	50.76±8.98	50.49±9.54	55.60±11.71
智能评分	42.48±7.01	52.42±9.35	51.02±8.64	55.00±10.10
Cohen’s d	0.23	−0.18	−0.06	0.05
组内相关系数	0.76**	0.80**	0.75**	0.78**

	广度	密度	乐观性	计划性	执行性	内控性	超越性
组内相关系数	0.28**	0.42**	0.41**	0.59**	0.28**	0.74**	0.28**

	广度	密度	乐观性	计划性	执行性	内控性	超越性
组内相关系数	0.28**	0.42**	0.41**	0.59**	0.28**	0.74**	0.28**

	总分2	广度2	密度2	乐观性2	计划性2	执行性2	内控性2	超越性2
总分1	0.79**	0.71**	0.63**	0.68**	0.63**	0.27**	0.66**	0.35**
广度1	0.77**	0.73**	0.63**	0.68**	0.62**	0.25**	0.64**	0.32**
密度1	0.77**	0.70**	0.67**	0.64**	0.64**	0.29**	0.66**	0.31**
乐观性1	0.69**	0.61**	0.47**	0.73**	0.48**	0.22**	0.54**	0.31**
计划性1	0.79**	0.74**	0.65**	0.66**	0.67**	0.26**	0.66**	0.30**
执行性1	0.81**	0.74**	0.68*	0.67**	0.69**	0.34**	0.72**	0.26**
内控性1	0.72**	0.64**	0.60**	0.61**	0.57**	0.27**	0.66**	0.28**
超越性1	0.35**	0.26**	0.23**	0.27*	0.21**	0.08	0.24**	0.49**