Deciding what to replicate: A formal definition of “replication value” and a decision model for replication study selection

Peder Isager, Robbie C.M. Van Aert, Štěpán Bahník, Mark J. Brandt, K. Andrew deSoto, Roger Giner-Sorolla, Joachim I. Krueger, Marco Perugini, Ivan Ropovik, Anna Elisabeth Van ’t Veer, Marek A. Vranka, Daniël Lakens

Research output: Working paperAcademic

Abstract

Robust scientific knowledge is contingent upon replication of original findings. However,researchers who conduct replication studies face a dicult problem; there are many morestudies in need of replication than there are funds available for replicating. To select studies forreplication eciently, we need to understand which studies arethe mostin need of replication.In other words, we need to understand which replication eorts have the highest expected utility.In this article we propose a general rule for study selection in replication research based onthereplication valueof the claims considered for replication. Thereplication valueof a claimis defined as the maximum expected utility we could gain by conducting a replication of theclaim, and is a function of (1) the value of being certain about the claim, and (2) uncertaintyabout the claim based on current evidence. We formalize this definition in terms of a causaldecision model, utilizing concepts from decision theory and causal graph modeling. We discussthe validity of usingreplication valueas a measure of expected utility gain, and we suggestapproaches for deriving quantitative estimates ofreplication value.
Original languageEnglish
DOIs
Publication statusSubmitted - 2020

Keywords

  • replication
  • causal model
  • expected utility
  • study selection

Fingerprint Dive into the research topics of 'Deciding what to replicate: A formal definition of “replication value” and a decision model for replication study selection'. Together they form a unique fingerprint.

Cite this