Preprint has been published in a journal as an article
Preprint / Version 1

Overcoming the “ostrich effect”

A narrative review on the incentives and consequences of questionable research practices in kinesiology


  • Nicholas Tiller Harbor-UCLA
  • Panteleimon Ekkekakis Michigan State University



ethics, exercise, health, methods, statistics, science


Increasing transparency and openness in science is an ongoing endeavor, one that has stimulated self-reflection and reform in many fields. However, kinesiology and its related disciplines are among those exhibiting an “ostrich effect” and a reluctance to acknowledge their methodological shortcomings. Notwithstanding several high-profile cases of scientific misconduct, scholars in the field are frequently engaged in questionable research practices (QRPs), such as biased experimental designs, inappropriate statistics, and dishonest/inexplicit reporting. To advance their careers, researchers are also “gaming the system” by manipulating citation metrics and publishing in predatory and/or pay-to-publish journals that lack robust peer-review. The consequences of QRPs in the discipline may be profound: from increasing the false positivity rate to eroding public trust in the very institutions tasked with informing public-health policy. But what are the incentives underpinning misconduct and QRPs? And what are the solutions? This narrative review is a consciousness raiser that explores i) the manifestations of QRPs in kinesiology; ii) the excessive publication pressures, funding pressures, and performance incentives that are likely responsible; and iii) possible solutions for reform.


Metrics Loading ...


Abt, G., Boreham, C., Davison, G., Jackson, R., Nevill, A., Wallace, E., & Williams, M. (2020). Power, precision, and sample size estimation in sport and exercise science research. Journal of Sports Sciences, 38(17), 1933–1935.

Abt, G., Jobson, S., Morin, J.-B., Passfield, L., Sampaio, J., Sunderland, C., & Twist, C. (2022). Raising the bar in sports performance research. Journal of Sports Sciences, 40(2), 125–129.

Altman, D. G., Gore, S. M., Gardner, M. J., & Pocock, S. J. (1983). Statistical guidelines for contributors to medical journals. British Medical Journal (Clinical Research Ed.), 286(6376), 1489–1493.

American Psychological Association. (2020). Publication manual of the American Psychological Association 2020: The official guide to APA style (7th ed.).

Andersen, M. B., McCullagh, P., & Wilson, G. J. (2007). But what do the numbers really tell us? Arbitrary metrics and effect size reporting in sport psychology research. Journal of Sport & Exercise Psychology, 29(5), 664–672.

Andrade, C. (2015). The primary outcome measure and its importance in clinical trials. The Journal of Clinical Psychiatry, 76(10), e1320-1323.

Aslam, A., Imanullah, S., Asim, M., & El-Menyar, A. (2013). Registration of clinical trials: Is it really needed? North American Journal of Medical Sciences, 5(12), 713–715.

Atkinson, G., & Batterham, A. M. (2015). True and false interindividual differences in the physiological response to an intervention. Experimental Physiology, 100(6), 577–588.

Atkinson, G., Loenneke, J. P., Fahs, C. A., Abe, T., & Rossow, L. M. (2015). Individual differences in the exercise-mediated blood pressure response: Regression to the mean in disguise? Clinical Physiology and Functional Imaging, 35(6), 490–491.

Atkinson, G., & Nevill, A. M. (2001). Selected issues in the design and analysis of sport performance research. Journal of Sports Sciences, 19(10), 811–827.

Atkinson, G., Williamson, P., & Batterham, A. M. (2019). Issues in the determination of “responders” and “non-responders” in physiological research. Experimental Physiology, 104(8), 1215–1225.

Bachynski, K. E., & Smoliga, J. M. (2021). Pseudomedicine for sports concussions in the USA. The Lancet. Neurology, 20(10), 791–792.

Barlow, P., Serôdio, P., Ruskin, G., McKee, M., & Stuckler, D. (2018). Science organisations and Coca-Cola’s “war” with the public health community: Insights from an internal industry document. Journal of Epidemiology and Community Health, 72(9), 761–763.

Baugh, F. (2002). Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably. Educational and Psychological Measurement, 62(2), 254–263.

Begley, C. G., & Ioannidis, J. P. A. (2015). Reproducibility in science: Improving the standard for basic and preclinical research. Circulation Research, 116(1), 116–126.

Bernard, C. (2019). Changing the way we report, interpret, and discuss our results to rebuild trust in our research. ENeuro, 6(4), ENEURO.0259-19.2019.

Bernards, J. R., Sato, K., Haff, G. G., & Bazyler, C. D. (2017). Current research and statistical practices in sport science and a need for change. Sports, 5(4), Article 4.

Bes-Rastrollo, M., Schulze, M. B., Ruiz-Canela, M., & Martinez-Gonzalez, M. A. (2013). Financial conflicts of interest and reporting bias regarding the association between sugar-sweetened beverages and weight gain: A systematic review of systematic reviews. PLoS Medicine, 10(12), e1001578; dicsussion e1001578.

Biagioli, M., & Lippman, A. (2020). Gaming the metrics: Misconduct and manipulation in academic research. MIT Press.

Bishop, D. (2008). An applied research model for the sport sciences. Sports Medicine (Auckland, N.Z.), 38(3), 253–263.

Bleakley, C., & MacAuley, D. (2002). The quality of research in sports journals. British Journal of Sports Medicine, 36(2), 124–125.

Blumenthal, J. A., Smith, P. J., Mabe, S., Hinderliter, A., Lin, P.-H., Liao, L., Welsh-Bohmer, K. A., Browndyke, J. N., Kraus, W. E., Doraiswamy, P. M., Burke, J. R., & Sherwood, A. (2019). Lifestyle and neurocognition in older adults with cognitive impairments: A randomized trial. Neurology, 92(3), e212–e223.

Bonafiglia, J. T., Islam, H., Preobrazenski, N., & Gurd, B. J. (2022). Risk of bias and reporting practices in studies comparing VO2max responses to sprint interval vs. continuous training: A systematic review and meta-analysis. Journal of Sport and Health Science, 11(5), 552–566.

Brainard, J. (2020). Articles in “predatory” journals receive few or no citations. Science (New York, N.Y.), 367(6474), 129.

Brischoux, F., & Angelier, F. (2015). Academia’s never-ending selection for productivity. Scientometrics, 103(1), 333–336.

Büttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020). Are questionable research practices facilitating new discoveries in sport and exercise medicine? The proportion of supported hypotheses is implausibly high. British Journal of Sports Medicine, 54(22), 1365–1371.

Caldwell, A. R., Vigotsky, A. D., Tenan, M. S., Radel, R., Mellor, D. T., Kreutzer, A., Lahart, I. M., Mills, J. P., Boisgontier, M. P., & Consortium for Transparency in Exercise Science (COTES) Collaborators. (2020). Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices. Sports Medicine (Auckland, N.Z.), 50(3), 449–459.

Callaway, E. (2011). Fraud investigation rocks Danish university. Nature.

Chahal, J., Tomescu, S. S., Ravi, B., Bach, B. R., Ogilvie-Harris, D., Mohamed, N. N., & Gandhi, R. (2012). Publication of sports medicine-related randomized controlled trials registered in The American Journal of Sports Medicine, 40(9), 1970–1977.

Chan, A.-W., Hróbjartsson, A., Jørgensen, K. J., Gøtzsche, P. C., & Altman, D. G. (2008). Discrepancies in sample size calculations and data analyses reported in randomised trials: Comparison of publications with protocols. BMJ (Clinical Research Ed.), 337, a2299.

Charles, P., Giraudeau, B., Dechartres, A., Baron, G., & Ravaud, P. (2009). Reporting of sample size calculation in randomised controlled trials: Review. BMJ (Clinical Research Ed.), 338, b1732.

Charter, R. A. (1997). Effect on measurement error on tests of statistical significance. Journal of Clinical and Experimental Neuropsychology, 19(3), 458–462.

Christensen, J. E., & Christensen, C. E. (1977). Statistical power analysis of health, physical education, and recreation research. Research Quarterly, 48(1), 204–208.

Colloca, L. (2019). The Placebo Effect in Pain Therapies. Annual Review of Pharmacology and Toxicology, 59, 191–211.

Coursol, A., & Wagner, E. E. (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice, 17(2), 136–137.

Cumming, G. (2008). Replication and p Intervals: P Values Predict the Future Only Vaguely, but Confidence Intervals Do Much Better. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 3(4), 286–300.

Cumming, G., Fidler, F., Kalinowski, P., & Lai, J. (2012). The statistical recommendations of the American psychological association publication manual: Effect sizes, confidence intervals, and meta‐analysis. Australian Journal of Psychology, 64, 138–146.

Curran-Everett, D. (2016). Explorations in statistics: Statistical facets of reproducibility. Advances in Physiology Education, 40(2), 248–252.

Curran-Everett, D., & Benos, D. J. (2007). Guidelines for reporting statistics in journals published by the American Physiological Society: The sequel. Advances in Physiology Education, 31(4), 295–298.

Dahlberg, J. E., & Mahler, C. C. (2006). The poehlman case: Running away from the truth. Science and Engineering Ethics, 12(1), 157–173.

Dalton, R. (2005). Obesity expert owns up to million-dollar crime. Nature, 434(7032), Article 7032.

Del Coso, J., Lara, B., Ruiz-Moreno, C., & Salinero, J. J. (2019). Challenging the Myth of Non-Response to the Ergogenic Effects of Caffeine Ingestion on Exercise Performance. Nutrients, 11(4), E732.

Desharnais, R., Jobin, J., Côté, C., Lévesque, L., & Godin, G. (1993). Aerobic exercise and the placebo effect: A controlled study. Psychosomatic Medicine, 55(2), 149–154.

Diels, J., Cunha, M., Manaia, C., Sabugosa-Madeira, B., & Silva, M. (2011). Association of financial or professional conflict of interest to research outcomes on health risks or nutritional assessment studies of genetically modified products. Food Policy, 36(2), 197–203.

Dinis-Oliveira, R. J., & Magalhães, T. (2016). The Inherent Drawbacks of the Pressure to Publish in Health Sciences: Good or Bad Science. F1000Research, 4, 419.

Diong, J., Butler, A. A., Gandevia, S. C., & Héroux, M. E. (2018). Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PloS One, 13(8), e0202121.

DuBois, J. M., Dueker, J. M., Anderson, E. E., & Campbell, J. (2008). The development and assessment of an NIH-funded research ethics training program. Academic Medicine: Journal of the Association of American Medical Colleges, 83(6), 596–603.

Dumas-Mallet, E., Button, K. S., Boraud, T., Gonon, F., & Munafò, M. R. (2017). Low statistical power in biomedical science: A review of three human research domains. Royal Society Open Science, 4(2), 160254.

Earnest, C. P., Roberts, B. M., Harnish, C. R., Kutz, J. L., Cholewa, J. M., & Johannsen, N. M. (2018). Reporting Characteristics in Sports Nutrition. Sports, 6(4), Article 4.

Edwards, M. A., & Roy, S. (2016). Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. Environmental Engineering Science, 34(1), 51–61.

Eisenmann, J. (2017). Translational Gap between Laboratory and Playing Field: New Era to Solve Old Problems in Sports Science. Translational Journal of the American College of Sports Medicine, 2(8), 37–43.

Ekkekakis, P., & Tiller, N. B. (2022). Extraordinary Claims in the Literature on High-Intensity Interval Training: II. Are the Extraordinary Claims Supported by Extraordinary Evidence? Kinesiology Review, 1(aop), 1–14.

Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. ELife, 10, e71601.

Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PloS One, 4(5), e5738.

Fanelli, D. (2010). “Positive” Results Increase Down the Hierarchy of the Sciences. PLOS ONE, 5(4), e10068.

Fanelli, D., Costas, R., & Larivière, V. (2015). Misconduct Policies, Academic Culture and Career Stage, Not Gender or Pressures to Publish, Affect Scientific Integrity. PloS One, 10(6), e0127556.

Fisher, R. A. (1926). The arrangement of field experiments. Journal of the Ministry of Agriculture, 33, 503–515.

Flanagin, A., Fontanarosa, P. B., & Bauchner, H. (2020). Preprints Involving Medical Research-Do the Benefits Outweigh the Challenges? JAMA, 324(18), 1840–1843.

Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505.

Freemantle, N. (2001). Interpreting the results of secondary end points and subgroup analyses in clinical trials: Should we lock the crazy aunt in the attic? BMJ, 322(7292), 989–991.

Gandevia, S. (2021). Publications, replication and statistics in physiology plus two neglected curves. The Journal of Physiology, 599(6), 1719–1721.

Gardner, M. J., Altman, D. G., Jones, D. R., & Machin, D. (1983). Is the statistical assessment of papers submitted to the “British Medical Journal” effective? Br Med J (Clin Res Ed), 286(6376), 1485–1488.

Garvican, L. A., Pottgiesser, T., Martin, D. T., Schumacher, Y. O., Barras, M., & Gore, C. J. (2011). The contribution of haemoglobin mass to increases in cycling performance induced by simulated LHTL. European Journal of Applied Physiology, 111(6), 1089–1101.

Gaspar, D. E. P., & Esteves, M. D. L. (2021). Awareness of the Misconduct in Sports Science Research. Annals of Applied Sport Science, 9(3), 0–0.

Gasparyan, A. Y., Ayvazyan, L., Akazhanov, N. A., & Kitas, G. D. (2014). Self-correction in biomedical publications and the scientific impact. Croatian Medical Journal, 55(1), 61–72.

Gerrits, R. G., Jansen, T., Mulyanto, J., Berg, M. J. van den, Klazinga, N. S., & Kringos, D. S. (2019). Occurrence and nature of questionable research practices in the reporting of messages and conclusions in international scientific Health Services Research publications: A structured assessment of publications authored by researchers in the Netherlands. BMJ Open, 9(5), e027903.

Gibbs, K. D., & Griffin, K. A. (2013). What Do I Want to Be with My PhD? The Roles of Personal Values and Structural Dynamics in Shaping the Career Interests of Recent Biomedical Science PhD Graduates. CBE Life Sciences Education, 12(4), 711–723.

Gibbs, N. M., & Gibbs, S. V. (2015). Misuse of “trend” to describe “almost significant” differences in anaesthesia research. British Journal of Anaesthesia, 115(3), 337–339.

Goodman, S. N. (1992). A comment on replication, P-values and evidence. Statistics in Medicine, 11(7), 875–879.

Gopalakrishna, G., Ter Riet, G., Vink, G., Stoop, I., Wicherts, J. M., & Bouter, L. M. (2022). Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands. PloS One, 17(2), e0263023.

Groenwold, R. H. H., & Dekkers, O. M. (2020). Measurement error in clinical research, yes it matters. European Journal of Endocrinology, 183(3), E3–E5.

Halperin, I., Vigotsky, A. D., Foster, C., & Pyne, D. B. (2018). Strengthening the Practice of Exercise and Sport-Science Research. International Journal of Sports Physiology and Performance, 13(2), 127–134.

Harris, J. D., Cvetanovich, G., Erickson, B. J., Abrams, G. D., Chahal, J., Gupta, A. K., McCormick, F. M., & Bach, B. R. (2014). Current status of evidence-based sports medicine. Arthroscopy: The Journal of Arthroscopic & Related Surgery: Official Publication of the Arthroscopy Association of North America and the International Arthroscopy Association, 30(3), 362–371.

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431.

Higginson, A. D., & Munafò, M. R. (2016). Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions. PLOS Biology, 14(11), e2000995.

Hopkins, W. G., Marshall, S. W., Batterham, A. M., & Hanin, J. (2009). Progressive Statistics for Studies in Sports Medicine and Exercise Science. Medicine & Science in Sports & Exercise, 41(1), 3–12.

How to spot dodgy academic journals. (2020). The Economist.

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.

Ioannidis, J. P. A., Boyack, K. W., & Baas, J. (2020). Updated science-wide author databases of standardized citation indicators. PLoS Biology, 18(10), e3000918.

Ioannidis, J. P. A., Ntzani, E. E., Trikalinos, T. A., & Contopoulos-Ioannidis, D. G. (2001). Replication validity of genetic association studies. Nature Genetics, 29(3), Article 3.

Islam, H., & Gurd, B. J. (2020). Exercise response variability: Random error or true differences in exercise response? Experimental Physiology, 105(12), 2022–2024.

John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychological Science, 23(5), 524–532.

Jones, S. R., Carley, S., & Harrison, M. (2003). An introduction to power and sample size estimation. Emergency Medicine Journal, 20(5), 453–458.

Kardeş, S., Levack, W., Özkuk, K., Atmaca Aydın, E., & Seringeç Karabulut, S. (2020). Retractions in Rehabilitation and Sport Sciences Journals: A Systematic Review. Archives of Physical Medicine and Rehabilitation, 101(11), 1980–1990.

Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review, 2(3), 196–217.

Khatra, O., Shadgan, A., Taunton, J., Pakravan, A., & Shadgan, B. (2021). A bibliometric analysis of the top cited articles in sports and exercise medicine. Orthopaedic Journal of Sports Medicine, 9(1), 2325967120969902.

Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biology, 14(5), e1002456.

King, E. G., Oransky, I., Sachs, T. E., Farber, A., Flynn, D. B., Abritis, A., Kalish, J. A., & Siracuse, J. J. (2018). Analysis of retracted articles in the surgical literature. American Journal of Surgery, 216(5), 851–855.

Kintisch, E. (2006). Poehlman Sentenced to 1 Year of Prison. Science.Org.

Kiss, A., Temesi, Á., Tompa, O., Lakner, Z., & Soós, S. (2021). Structure and trends of international sport nutrition research between 2000 and 2018: Bibliometric mapping of sport nutrition science. Journal of the International Society of Sports Nutrition, 18(1), 12.

Klimentidis, Y. C., Raichlen, D. A., Bea, J., Garcia, D. O., Wineinger, N. E., Mandarino, L. J., Alexander, G. E., Chen, Z., & Going, S. B. (2018). Genome-wide association study of habitual physical activity in over 377,000 UK Biobank participants identifies multiple variants including CADM2 and APOE. International Journal of Obesity (2005), 42(6), 1161–1176.

Knudson, D. (2009). Significant and meaningful effects in sports biomechanics research. Sports Biomechanics, 8(1), 96–104.

Knudson, D. (2012). Twenty-year trends of authorship and sampling in applied biomechanics research. Perceptual and Motor Skills, 114(1), 16–20.

Knudson, D. (2017a). Twenty Years of Authorship, Sampling, and References in Kinesiology Research Reports. International Journal of Kinesiology in Higher Education, 1(2), 44–52.

Knudson, D. (2017b). Confidence crisis of results in biomechanics research. Sports Biomechanics, 16(4), 425–433.

Knudson, D. (2019). Judicious Use of Bibliometrics to Supplement Peer Evaluations of Research in Kinesiology in: Kinesiology Review Volume 8 Issue 2 (). Kinesiology Review, 8(2), 100–109.

Kondro, W. (2005). Université de Montréal in the dark about fraud. CMAJ, 172(10), 1278-1278-a.

Krans, B. (2022). Lawsuit: Coca-Cola Fake Ads About Obesity. Healthline.Com.

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863.

Landhuis, E. (2016). Scientific literature: Information overload. Nature, 535(7612), Article 7612.

Lesser, L. I., Ebbeling, C. B., Goozner, M., Wypij, D., & Ludwig, D. S. (2007). Relationship between funding source and conclusion among nutrition-related scientific articles. PLoS Medicine, 4(1), e5.

Lohse, K. R., Sainani, K. L., Taylor, J. A., Butson, M. L., Knight, E. J., & Vickers, A. J. (2020). Systematic review of the use of “magnitude-based inference” in sports science and medicine. PLOS ONE, 15(6), e0235318.

Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. Science (New York, N.Y.), 355(6325), 584–585.

Lucas, M. (2015). Conflicts of interest in nutritional sciences: The forgotten bias in meta-analysis. World Journal of Methodology, 5(4), 175–178.

Maggio, L., Dong, T., Driessen, E., & Artino, A. (2019). Factors associated with scientific misconduct and questionable research practices in health professions education. Perspectives on Medical Education, 8(2), 74–82.

Mallapaty, S. (2020). China bans cash rewards for publishing papers. Nature, 579(7797), 18.

Marticorena, F. M., Carvalho, A., Oliveira, L. F. D., Dolan, E., Gualano, B., Swinton, P., & Saunders, B. (2021). Nonplacebo Controls to Determine the Magnitude of Ergogenic Interventions: A Systematic Review and Meta-analysis. Medicine and Science in Sports and Exercise, 53(8), 1766–1777.

Martin, J. J. (2020). Grants: The Good, the Bad, the Ugly, and the Puzzling. Kinesiology Review, 10(1), 18–28.

Mattioni Maturana, F., Martus, P., Zipfel, S., & NIEß, A. M. (2021). Effectiveness of HIIE versus MICT in Improving Cardiometabolic Risk Factors in Health and Disease: A Meta-analysis. Medicine and Science in Sports and Exercise, 53(3), 559–573.

Montero, D., & Lundby, C. (2017). Refuting the myth of non‐response to exercise training: ‘Non‐responders’ do respond to higher dose of training. The Journal of Physiology, 595(11), 3377–3387.

Moylan, E. C., & Kowalczuk, M. K. (2016). Why articles are retracted: A retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open, 6(11), e012047.

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., du Sert, N. P., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021.

Murphy, J., Mesquida, C., Caldwell, A. R., Earp, B. D., & Warne, J. P. (2023). Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science. Sports Medicine (Auckland, N.Z.).

Nagele, P. (2003). Misuse of standard error of the mean (sem) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. BJA: British Journal of Anaesthesia, 90(4), 514–516.

Nosek, B. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.

Nosek, B. A., & Bar-Anan, Y. (2012). Scientific Utopia: I. Opening scientific communication. ArXiv:1205.1055 [Physics].

Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(6), 615–631.

Open Science Collaboration. (2015). PSYCHOLOGY. Estimating the reproducibility of psychological science. Science (New York, N.Y.), 349(6251), aac4716.

Pabst, S., Brand, M., & Wolf, O. T. (2013). Stress effects on framed decisions: There are differences for gains and losses. Frontiers in Behavioral Neuroscience, 7, 142.

Padilla, J., Leary, E., & Limberg, J. K. (2021). Identifying responders versus non-responders: Incorporation of controls is required for sound statistical inference. Experimental Physiology, 106(2), 375–376.

Prom-Wormley, E., Adkins, A., Waldman, I. D., & Dick, D. (2017). Critical Issues in Genetic Association Studies. In Psychological Science Under Scrutiny (pp. 221–249). John Wiley & Sons, Ltd.

Quan, W., Chen, B., & Shu, F. (2017). Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016). Aslib Journal of Information Management, 69(5), 486–502.

Robertson, P. S. (2020). Man & machine: Adaptive tools for the contemporary performance analyst. Journal of Sports Sciences, 38(18), 2118–2126.

Sabroe, I., Dockrell, D. H., Vogel, S. N., Renshaw, S. A., Whyte, M. K. B., & Dower, S. K. (2007). Identifying and hurdling obstacles to translational research. Nature Reviews. Immunology, 7(1), 77–82.

Sainani, K., & Chamari, K. (2022). Wish List for Improving the Quality of Statistics in Sport Science. International Journal of Sports Physiology and Performance, 17(5), 673–674.

Sainani, K. L., Borg, D. N., Caldwell, A. R., Butson, M. L., Tenan, M. S., Vickers, A. J., Vigotsky, A. D., Warmenhoven, J., Nguyen, R., Lohse, K. R., Knight, E. J., & Bargary, N. (2021). Call to increase statistical collaboration in sports science, sport and exercise medicine and sports physiotherapy. British Journal of Sports Medicine, 55(2), 118–122.

Sarabipour, S., Debat, H. J., Emmott, E., Burgess, S. J., Schwessinger, B., & Hensel, Z. (2019). On the value of preprints: An early career researcher perspective. PLoS Biology, 17(2), e3000151.

Scheel, A. M., Schijen, M., & Lakens, D. (2020). An excess of positive results: Comparing the standard Psychology literature with Registered Reports. PsyArXiv.

Schmid, S. L. (2017). Five years post-DORA: Promoting best practices for research assessment. Molecular Biology of the Cell, 28(22), 2941–2944.

Schulz, K. F., Altman, D. G., Moher, D., & CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. BMC Medicine, 8, 18.

Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. L. (2022). Reporting and transparent research practices in sports medicine and orthopaedic clinical trials: A meta-research study. BMJ Open, 12(8), e059347.

Serodio, P., Ruskin, G., McKee, M., & Stuckler, D. (2020). Evaluating Coca-Cola’s attempts to influence public health ‘in their own words’: Analysis of Coca-Cola emails with public health academics leading the Global Energy Balance Network. Public Health Nutrition, 23, 1–7.

Serra-Garcia, M., & Gneezy, U. (2021). Nonreplicable publications are cited more than replicable ones. Science Advances, 7(21), eabd1705.

Shen, C., & Björk, B.-C. (2015). “Predatory” open access: A longitudinal study of article volumes and market characteristics. BMC Medicine, 13, 230.

Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., … Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356.

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366.

Smaldino, P. E., & McElreath, R. (2016). The Natural Selection of Bad Science. Royal Society Open Science, 3(9), 160384.

Smoliga, J. M. (2020). Interpreting Biomarker Data After Concussion and Repeated Subconcussive Head Impacts: Challenges in Evaluating Brain Protection. JAMA Neurology, 77(12), 1477–1478.

Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021). Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour, 5(8), Article 8.

Sox, H. C., & Rennie, D. (2006). Research Misconduct, Retraction, and Cleansing the Medical Literature: Lessons from the Poehlman Case. Annals of Internal Medicine, 144(8), 609–613.

Spence, J. C., & Blanchard, C. (2001). Publication bias in sport and exercise psychology: The games we play. International Journal of Sport Psychology, 32(4), 386–399.

Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018). What meta-analyses reveal about the replicability of psychological research. Psychological Bulletin, 144(12), 1325–1346.

Statista. (2020). Global sports nutrition & supplement market 2025. Statista.

Steen, R. G. (2011). Misinformation in the medical literature: What role do error and fraud play? Journal of Medical Ethics, 37(8), 498–503.

Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995). Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa. The American Statistician, 49(1), 108–112.

Stuckler, D., Ruskin, G., & McKee, M. (2018). Complexity and conflicts of interest statements: A case-study of emails exchanged between Coca-Cola and the principal investigators of the International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE). Journal of Public Health Policy, 39(1), 49–56.

Teo, Y. Y. (2008). Common statistical issues in genome-wide association studies: A review on power, data quality control, genotype calling and population structure. Current Opinion in Lipidology, 19(2), 133–143.

The Society for Transparency, Openness, and Replication in Kinesiology. (n.d.). Reports in Sport and Exercise. Retrieved March 3, 2022, from

Thomas, J. R., Salazar, W., & Landers, D. M. (1991). What is missing in p less than .05? Effect size. Research Quarterly for Exercise and Sport, 62(3), 344–348.

Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26–30.

Tijdink, J. K., Verbeke, R., & Smulders, Y. M. (2014). Publication Pressure and Scientific Misconduct in Medical Scientists. Journal of Empirical Research on Human Research Ethics, 9(5), 64–71.

Tiller, N. B. (2020). The Skeptic’s Guide to Sports Science: Confronting Myths of the Health and Fitness Industry. Routledge.

Tiller, N. B., Sullivan, J. P., & Ekkekakis, P. (2022). Baseless Claims and Pseudoscience in Health and Wellness: A Call to Action for the Sports, Exercise, and Nutrition-Science Community. Sports Medicine (Auckland, N.Z.).

Twomey, R., Yingling, V., Warne, J., Schneider, C., McCrum, C., Atkins, W., Murphy, J., Medina, C. R., Harlley, S., & Caldwell, A. (2021). The Nature of Our Literature: A Registered Report on the Positive Result Rate and Reporting Practices in Kinesiology. Communications in Kinesiology, 1(3), Article 3.

Vagenas, G., Palaiothodorou, D., & Knudson, D. (2018). Thirty-year Trends of Study Design and Statistics in Applied Sports and Exercise Biomechanics Research. International Journal of Exercise Science, 11(1), 239–259.

van de Schoot, R., Winter, S. D., Griffioen, E., Grimmelikhuijsen, S., Arts, I., Veen, D., Grandfield, E. M., & Tummers, L. G. (2021). The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum. Frontiers in Psychology, 12, 621547.

Van Noorden, R. (2010). Metrics: A profusion of measures. Nature, 465(7300), 864–866.

Van Noorden, R., & Singh Chawla, D. (2019). Hundreds of extreme self-citing scientists revealed in new database. Nature, 572(7771), 578–579.

Vankov, I., Bowers, J., & Munafò, M. R. (2014). Article Commentary: On the Persistence of Low Power in Psychological Science. Quarterly Journal of Experimental Psychology, 67(5), 1037–1040.

Viken, H., Reitlo, L. S., Zisko, N., Nauman, J., Aspvik, N. P., Ingebrigtsen, J. E., Wisløff, U., & Stensvold, D. (2019). Predictors of Dropout in Exercise Trials in Older Adults: The Generation 100 Study. Medicine & Science in Sports & Exercise, 51(1), 49–55.

Walters, J. (2015, August 11). Nutrition experts alarmed by nonprofit downplaying role of junk food in obesity. The Guardian.

Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133.

Watanabe, R. M. (2011). Statistical issues in gene association studies. Methods in Molecular Biology (Clifton, N.J.), 700, 17–36.

Wiedermann, C. J. (2016). Ethical publishing in intensive care medicine: A narrative review. World Journal of Critical Care Medicine, 5(3), 171–179.

Williams, C. J., Li, Z., Harvey, N., Lea, R. A., Gurd, B. J., Bonafiglia, J. T., Papadimitriou, I., Jacques, M., Croci, I., Stensvold, D., Wisloff, U., Taylor, J. L., Gajanand, T., Cox, E. R., Ramos, J. S., Fassett, R. G., Little, J. P., Francois, M. E., Hearon, C. M., … Coombes, J. S. (2021). Genome wide association study of response to interval and continuous exercise training: The Predict-HIIT study. Journal of Biomedical Science, 28(1), 37.

Wood, J., Freemantle, N., King, M., & Nazareth, I. (2014). Trap of trends to statistical significance: Likelihood of near significant P value becoming more significant with extra data. BMJ, 348, g2215.

Yong, E. (2012). Nobel laureate challenges psychologists to clean up their act. Nature.