• Jenna A Harder.
• Department of Psychology, Michigan State University.
• Perspect Psychol Sci. 2020 Sep 1; 15 (5): 1158-1177.

AbstractWhen analyzing data, researchers may have multiple reasonable options for the many decisions they must make about the data-for example, how to code a variable or which participants to exclude. Therefore, there exists a multiverse of possible data sets. A classic multiverse analysis involves performing a given analysis on every potential data set in this multiverse to examine how each data decision affects the results. However, a limitation of the multiverse analysis is that it addresses only data cleaning and analytic decisions, yet researcher decisions that affect results also happen at the data-collection stage. I propose an adaptation of the multiverse method in which the multiverse of data sets is composed of real data sets from studies varying in data-collection methods of interest. I walk through an example analysis applying the approach to 19 studies on shooting decisions to demonstrate the usefulness of this approach and conclude with a further discussion of the limitations and applications of this method.

17 keywords...

hide…