Studies of Psychology and Behavior ›› 2017, Vol. 15 ›› Issue (5): 577-586.

    Next Articles

A Crude Evaluation on the Two Roots of the Reproducibility Crisis in Psychology

LUO Dasen   

  1. Indiana University of Pennsylvania, USA
  • Received:2017-08-10 Online:2017-12-28 Published:2017-09-20

心理学可重复性危机两种根源的评估

骆大森   

  1. 美国宾夕法尼亚印第安纳大学, 美国
  • 通讯作者: 骆大森,E-mail:dluo@iup.edu

Abstract: The reproducibility crisis in psychology is known to have two roots, the root in the traditional statistical system of null hypothesis significance testing, and that in the academic tradition of psychology. This article was an attempt to crudely estimate the respective impacts of the two roots on the reproducibility crisis in psychology. The results reported by Open Science Collaboration (2015) were analyzed using the methods suggested by Goodman (1992) and by Cumming (2008) to roughly estimate the limiting influence on reproducibility imposed by the traditional system of statistics. The estimated limiting influence, although quite notable, appears to be far short of being able to account for the reproducibility rate as low as 36% indicated by the report, suggesting that factors other than the traditional system of statistics have played a tremendous role in the crisis. The model proposed by Ioannidis (2005) was adopted to analyze the possible impacts of factors other than the traditional system of statistics, and possible ranges of the joint impact of bias and the probability of true alternative hypotheses were extrapolated。 The analysis led to estimates indicating that, of all original positive results, only no more than one third, and probably even less, was true positive, and a considerable portion of these positive results was caused by bias. These results may help explicate how these factors are likely to contribute to the current crisis.

Key words: the reproducibility crisis in psychology, limitations of traditional system of statistical testing, flaws in the academic tradition of psychology, the probability of true alternative hypotheses, bias, evaluation on the roots of the reproducibility crisis

摘要: 心理学的可重复性危机有两大已知的根源:传统统计学中虚无假设显著性检验体系的局限,和心理学的学术传统中的弊端,本文以开放科学协作组2015年报告的数据为依据,试对这两个根源的影响作一粗略的估算。采用Goodman(1992)和Cumming(2008)提出的方法对传统统计体系所加诸于可重复性的限制加以分析后,估算的结果表明传统统计学体系的制约,虽然有举足轻重的影响,却远不能完全解释该报告中低至36%的可重复率,该报告所反映的状况,显然还另有重大的非统计学的根源。本文进一步用Ioannidis(2005)提出的模型对这类非统计学因素的影响加以分析。分析后得到的若干组人为偏差率和Ha真实概率的估算,表明在原来研究所获得的几乎清一色的阳性结果中,大约只有不到三分之一或更低的比例是真阳性,而且相当部分的阳性结果,可能由人为偏差所造成。这样的分析可比较具体地描述该类因素对当前可重复性危机的可能影响。

关键词: 心理学可重复性危机, 传统统计检验的局限, 心理学学术传统中的弊端, 备择假设真实概率, 人为偏差, 危机根源的评估

CLC Number: