Exploratory Research in Sport and Exercise Science
DOI:
https://doi.org/10.51224/SRXIV.457Keywords:
hypothesis testing, confirmatory, data analysis, error control, theory driven analysis, questionable research practicesAbstract
Quantitative exploratory research implies a flexible examination of a dataset with the purpose of finding patterns, associations, and interactions between variables to help formulate a hypothesis, which should then be severely tested in a future confirmatory study. In many fields, including sport and exercise science, exploratory research is not openly reported. At the same time, experts agree that most of the research we conduct is indeed exploratory, and that exploration is a crucial step in scientific knowledge generation. Using a flowchart, we review how data are typically collected and used, and we distinguish exploratory from confirmatory studies by arguing that data-driven analyses, where the Type I and Type II error rate cannot be controlled, is what characterises exploratory research. Even if a study tests a hypothesis in an error-controlled manner, often it also includes exploratory analyses on the data. We ask which factors increase the quality and value of exploratory analyses, and highlight large sample sizes, uncommon sample compositions, rigorous data collection, widely used measures, observing a logical and coherent pattern across multiple variables, and the potential for generating new research questions as the main factors. Finally, we provide guidelines for carrying out and transparently writing up an exploratory study.
Metrics
References
Australian Academy of Science. (2022). Getting Ahead of the Game: Athlete Data in Professional Sport. www.science.org.au/datainsport
Bandalos, D. L., & Finney, S. J. (2018). Factor Analysis. Exploratory and Confirmatory. In G. R. Hancock, L. M. Stapleton, & R. O. Mueller (Eds.), The Reviewer’s Guide to Quantitative Methods in the Social Sciences (pp. 98-122). Routdlege.
Bishop, D. (2008). An applied research model for the sport sciences. Sports Medicine, 38(3), 253-263. https://doi.org/10.2165/00007256-200838030-00005
Cortex. (2021). Exploratory reports - author guidelines. Retrieved 12 April from https://legacyfileshare.elsevier.com/promis_misc/Exploratory_Reports_Guidelines.pdf
Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation, 10(7).
de Groot, A. D. (2014). The meaning of "significance" for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas]. Acta Psychologica, 148, 188-194. https://doi.org/10.1016/j.actpsy.2014.02.001
DeBruine, L., & Lakens, D. (2024). papercheck: Check Scientific Papers for Best Practices. R package version 0.0.0.9014. Retrieved 28 May from https://scienceverse.github.io/papercheck/, https://github.com/debruine/papercheck.
Derlatka, M., & Parfieniuk, M. (2023). Real-world measurements of ground reaction forces of normal gait of young adults wearing various footwear. Scientific Data, 10(1), 60. https://doi.org/10.1038/s41597-023-01964-z
Dirnagl, U. (2020). Preregistration of exploratory research: Learning from the golden age of discovery. Plos Biology, 18(3), e3000690. https://doi.org/10.1371/journal.pbio.3000690
Figueiredo, L., Scherer, C., & Cabral, J. S. (2022). A simple kit to use computational notebooks for more openness, reproducibility, and productivity in research. PLoS Computational Biology, 18(9), e1010356. https://doi.org/10.1371/journal.pcbi.1010356
Francis, G. (2017). Equivalent statistics and data interpretation. Behavior Research Methods, 49(4), 1524-1538. https://doi.org/10.3758/s13428-016-0812-3
Gaus, W., Mayer, B., & Muche, R. (2015). Interpretation of statistical significance-exploratory versus confirmative testing in clinical trials, epidemiological studies, meta-analyses and toxicological screening (using Ginkgo biloba as an example). Clininical & Experimental Pharmacology, 5(4), 1000182.
Haugen, T. A., Breitschädel, F., Wiig, H., & Seiler, S. (2021). Countermovement Jump Height in National-Team Athletes of Various Sports: A Framework for Practitioners and Scientists. International Journal of Sports Physiology and Performance, 16(2), 184-189. https://doi.org/10.1123/ijspp.2019-0964
Höfler, M., McDonald, B., Kanske, P., & Miller, R. (2023). Means to valuable exploration II: How to explore data to modify existing claims and create new ones. Meta-Psychology, 7, MP.2022.3270
Höfler, M., Scherbaum, S., Kanske, P., McDonald, B., & Miller, R. (2022). Means to valuable exploration: I. The blending of confirmation and exploration and how to resolve it. Meta-Psychology, 6, MP.2021.2837
Hussain, S. J., & Cohen, L. G. (2017). Exploratory studies: a crucial step towards better hypothesis-driven confirmatory research in brain stimulation. Journal of Physiology, 595(4), 1013-1014. https://doi.org/10.1113/JP273582
ICH. (1998). Statistical principles for clinical trials (E9). International Council for harmonisation of technical requirements for pharmaceuticals for human use. Retrieved 15 April from https://database.ich.org/sites/default/files/E9_Guideline.pdf
ICH. (2022). General considerations for clinical studies E8(R1). International Council for harmonisation of technical requirements for pharmaceuticals for human use. Retrieved 15 April from https://database.ich.org/sites/default/files/ICH_E8-R1_Guideline_Step4_2021_1006.pdf
IRSP. (2018). Exploratory Reports at IRSP: Guidelines for Authors. Retrieved 06 May from https://rips-irsp.com/exploratory-reports#exploratory-reports-at-irsp-guidelines-for-authors
Janiszewski, C., & van Osselaer, S. M. J. (2021). The benefits of candidly reporting consumer research. Journal of Consumer Psychology, 31(4), 633-646.
Jauhiainen, S., Pohl, A. J., Ayramo, S., Kauppi, J. P., & Ferber, R. (2020). A hierarchical cluster analysis to determine whether injured runners exhibit similar kinematic gait patterns. Scandinavian Journal of Medicine & Science in Sports, 30(4), 732-740. https://doi.org/10.1111/sms.13624
Kimmelman, J., Mogil, J. S., & Dirnagl, U. (2014). Distinguishing between Exploratory and Confirmatory Preclinical Research Will Improve Translation. PLoS Biology 12(5): e1001863. https://doi.org/10.1371/journal.pbio.1001863
Kirk, C. (2024). A 5-Year Analysis of Age, Stature and Armspan in Mixed Martial Arts. Reserch Quarterly for Exercise and Sport, 95(2), 450-457. https://doi.org/10.1080/02701367.2023.2252473
Kross, S., Carchedi, N., Bauer, B., & Grdina, G. (2020). _swirl: Learn R, in R_. R package version 2.4.5. Retrieved 26 April 2024 from https://CRAN.R-project.org/package=swirl
Lakens, D. (2019). The value of preregistration for psychological science: A conceptual analysis. Japanese Phsychological Review, 62(3), 221-230.
Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1), 33267.
Lakens, D. (2024). When and How to Deviate from a Preregistration. Collabra: Psychology, 10(1).
Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024). The benefits of preregistration and Registered Reports. Evidence-Based Toxicology, 2(1), 2376046. https://doi.org/10.1080/2833373X.2024.2376046
Luijken, K., Dekkers, O. M., Rosendaal, F. R., & Groenwold, R. H. H. (2022). Exploratory analyses in aetiologic research and considerations for assessment of credibility: mini-review of literature. British Medical Journal, 377, e070111. https://doi.org/10.1136/bmj-2021-070113
Mayo, D. G. (2018). How Not to Corrupt Power. In D. G. Mayo (Ed.), Statistical Inference as Severe Testing (pp. 353-370). Cambridge University Press.
McIntosh, R. D. (2017). Exploratory reports: A new article type for Cortex. Cortex, 96, A1-A4. https://doi.org/10.1016/j.cortex.2017.07.014
McPhetres, J. (2020). What should a preregistration contain? Retrieved 29 April from https://osf.io/preprints/psyarxiv/cj5mh
Mesquida, C., Murphy, J., Lakens, D., & Warne, J. (2022). Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field. Royal Society Open Science, 9(12), 220946. https://doi.org/10.1098/rsos.220946
Mogil, J. S., & Macleod, M. R. (2017). No publication without confirmation. Nature, 542(7642), 409-411. https://doi.org/DOI 10.1038/542409a
Munafò, M. R., & Smith, G. D. (2018). Repeating experiments is not enough. Nature, 553(7689), 399-401.
Murphy, J., Mesquida, C., Caldwell, A. R., Earp, B. D., & Warne, J. P. (2023). Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science. Sports Medicine, 53(1), 281-291. https://doi.org/10.1007/s40279-022-01749-1
Nilsen, E. B., Bowler, D. E., & Linnell, J. D. C. (2020). Exploratory and confirmatory research in the open science era. Journal of Applied Ecology, 57(4), 842-847. https://doi.org/10.1111/1365-2664.13571
Nordmann, E., & DeBruine, L. (2023). Applied Data Skills. v3.0. Retrieved 19 June from https://psyteachr.github.io/ads-v3/
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences of the United States of America, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114
Olken, B. A. (2015). Promises and perils of pre-analysis plans. Journal of Economic Perspectives, 29(3), 61-80.
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hrobjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S.,…Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. British Medical Journal, 372, n71. https://doi.org/10.1136/bmj.n71
Pearson, R. (2011). Exploring Data in Engineering, the Sciences, and Medicine. Oxford University Press.
Perroni, F., Castagna, C., Amatori, S., Gobbi, E., Vetrano, M., Visco, V., Guidetti, L., Baldari, C., Rocchi, M. B. L., & Sisti, D. (2023). Use of Exploratory Factor Analysis to Assess the Fitness Performance of Youth Football Players. Journal of Strength and Conditioning Research, 37(7), E430-E437. https://doi.org/10.1519/Jsc.0000000000004414
Pham, M. T., & Oh, T. T. (2021). Preregistration is neither sufficient nor necessary for good science. Journal of Consumer Psychology, 31(1), 163-176.
Ramírez-López, C., Till, K., Boyd, A., Bennet, M., Piscione, J., Bradley, S., Giuliano, P., Leduc, C., & Jones, B. (2021). Coopetition: cooperation among competitors to enhance applied research and drive innovation in elite sport. British Journal of Sports Medicine, 55(10), 519–530.
Reichenbach, H. (1938). The three tasks of epistemology. In Experience and prediction: An analysis of the foundations and the structure of knowledge (pp. 3-16). University of Chicago Press.
Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021). Why Hypothesis Testers Should Spend Less Time Testing Hypotheses. Perspectives on Psychological Science, 16(4), 744-755, 1745691620966795. https://doi.org/10.1177/1745691620966795
Schwab, S., & Held, L. (2020). Different worlds. Confirmatory versus exploratory research. Significance, 17(2), 8-9.
Shell, E. R. (2016). Hurdling obstacles: Meet Marcia McNutt, Scientist, Administrator, Editor, and Now National Academy of Sciences President. Science, 353, 116-119.
Staunton, C. A., Romann, M., Björklund, G., & Born, D. P. (2024). Diving into a pool of data: Using principal component analysis to optimize performance prediction in women's short-course swimming. Journal of Sports Sciences, 42(6), 519-526. https://doi.org/10.1080/02640414.2024.2346670
Suppiah, H. T., Gastin, P. B., & Driller, M. W. (2022). A Strategy to Inform Athlete Sleep Support From Questionnaire Data and Its Application in an Elite Athlete Cohort. International Journal of Sports Physiology and Performance, 17(10), 1532-1536. https://doi.org/10.1123/ijspp.2021-0561
Tong, C. (2019). Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science. American Statistician, 73, 246-261. https://doi.org/10.1080/00031305.2018.1518264
Tukey, J. W. (1980). We Need Both Exploratory and Confirmatory. American Statistician, 34(1), 23-25. https://doi.org/10.2307/2682991
Tunç, D. U., Tunç, M. N., & Lakens, D. (2023). The epistemic and pragmatic function of dichotomous claims based on statistical hypothesis tests. Theory & Psychology, 33(3), 403-423. https://doi.org/10.1177/09593543231160112
Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L., & Kievit, R. A. (2012). An Agenda for Purely Confirmatory Research. Perspectives on Psychological Science, 7(6), 632-638. https://doi.org/10.1177/1745691612463078
Watkins, M. W. (2018). Exploratory Factor Analysis: A Guide to Best Practice. Journal of Black Psychology, 44(3), 219-246. https://doi.org/10.1177/0095798418771807
Weaving, D., Beggs, C., Dalton-Barron, N., Jones, B., & Abt, G. (2019). Visualizing the Complexity of the Athlete-Monitoring Cycle Through Principal-Component Analysis. International Journal of Sports Physiology and Performance, 14(9), 1304-1310. https://doi.org/10.1123/ijspp.2019-0045
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J. W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R.,…Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18
Downloads
Posted
Categories
License
Copyright (c) 2024 Massimiliano Ditroilo, Cristian Mesquida, Grant Abt, Daniël Lakens (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
