An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports

Anne M. Scheel (Corresponding author), Mitchell R.M.J. Schijen, Daniël Lakens

Research output: Contribution to journalArticleAcademicpeer-review

49 Citations (Scopus)
35 Downloads (Pure)

Abstract

Selectively publishing results that support the tested hypotheses (“positive” results) distorts the available evidence for scientific claims. For the past decade, psychological scientists have been increasingly concerned about the degree of such distortion in their literature. A new publication format has been developed to prevent selective reporting: In Registered Reports (RRs), peer review and the decision to publish take place before results are known. We compared the results in published RRs (N = 71 as of November 2018) with a random sample of hypothesis-testing studies from the standard literature (N = 152) in psychology. Analyzing the first hypothesis of each article, we found 96% positive results in standard reports but only 44% positive results in RRs. We discuss possible explanations for this large difference and suggest that a plausible factor is the reduction of publication bias and/or Type I error inflation in the RR literature.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalAdvances in Methods and Practices in Psychological Science
Volume4
Issue number2
DOIs
Publication statusPublished - 1 Apr 2021

Bibliographical note

Publisher Copyright:
© The Author(s) 2021.

Keywords

  • hypothesis testing
  • open data
  • preregistered
  • publication bias
  • Registered Reports

Fingerprint

Dive into the research topics of 'An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports'. Together they form a unique fingerprint.

Cite this