BACKGROUND Confirmatary or exploratory? Most research lies on a continuum between the two types

1. Merit of pre-registration YongQi Cong and Sam Beekhuizen – University of Amsterdam • BACKGROUND • Confirmatary or exploratory? • Most research lies on a continuum between the two types • Most research is exploratory, while most statistical tests are meant for confirmatory purposes • When the data inspire a hypothesis, you cannot use the same data to test that hypothesis • Confirmation bias and hindsight bias • Quality vs Quantity • SOLUTION = • PRE-REGISTRATION • What is it? • Designand analysis of the study is defined beforehand, as opposed to posthoc analysis. • Types: • Regular, Public, With adversarial • collaboration • Discussion • Researchers cannot be trusted to observe the distinction of exploratory vs confirmatory study if they are not forced to (?) • Two stages of research: • Fishing expedition  confirm • There’s nothing wrong with exploratory research as long as it is acknowledged as such • Shortcut: • Collect all data at once, and split • them into subsets • The extra work involved in pre-registering an experiment is a small price to pay for a large increase in evidence impact. • Danger: reverse research order: • Research  pre-registration •  confirm results • This is FRAUD! Clear answers need clear questions! • Advantage of pre-registration • Clear distinction of confirmataryvsexplorotary research • Prevent QRPs • Easier for replication • Pre-registration replication reports makes publication easier References Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on PsychologicalScience, 7(6), 543-554. Chambers, C. D., Feredoes, E., Muthukumaraswamy, S. D., & Etchells, P. (2014). Instead of" playing the game" it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond. AIMS Neuroscience, 1(1), 4-17. Wagenmakers, E.–J. (2014). Study pre-registration. Presentation for the conference “Improving Scientific Practice: Dealing with the Human Factors ”, University of Amsterdam, Amsterdam, the Netherlands, September 2014. Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7(6), 632-638.