Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints

Peder Isager, Robbie C.M. Van Aert, Štěpán Bahník, Mark J. Brandt, K. Andrew DeSoto, Roger Giner-Sorolla, Joachim I. Krueger, Marco Perugini, Ivan Ropovik, Anna Elisabeth Van ’t Veer, Marek A. Vranka, Daniël Lakens

Research output: Working paperAcademic

4 Downloads (Pure)


Robust scientific knowledge is contingent upon replication of original findings. However,researchers who conduct replication studies face a dicult problem; there are many morestudies in need of replication than there are funds available for replicating. To select studies forreplication eciently, we need to understand which studies arethe mostin need of replication.In other words, we need to understand which replication eorts have the highest expected utility.In this article we propose a general rule for study selection in replication research based onthereplication valueof the claims considered for replication. Thereplication valueof a claimis defined as the maximum expected utility we could gain by conducting a replication of theclaim, and is a function of (1) the value of being certain about the claim, and (2) uncertaintyabout the claim based on current evidence. We formalize this definition in terms of a causaldecision model, utilizing concepts from decision theory and causal graph modeling. We discussthe validity of usingreplication valueas a measure of expected utility gain, and we suggestapproaches for deriving quantitative estimates ofreplication value.
Original languageEnglish
Number of pages16
Publication statusPublished - 2 Sep 2020


  • replication
  • causal model
  • expected utility
  • study selection


Dive into the research topics of 'Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints'. Together they form a unique fingerprint.

Cite this