Overcoming the “ostrich effect”
A narrative review on the incentives and consequences of questionable research practices in kinesiology
DOI:
https://doi.org/10.51224/SRXIV.273Keywords:
ethics, exercise, health, methods, statistics, scienceAbstract
Increasing transparency and openness in science is an ongoing endeavor, one that has stimulated self-reflection and reform in many fields. However, kinesiology and its related disciplines are among those exhibiting an “ostrich effect” and a reluctance to acknowledge their methodological shortcomings. Notwithstanding several high-profile cases of scientific misconduct, scholars in the field are frequently engaged in questionable research practices (QRPs), such as biased experimental designs, inappropriate statistics, and dishonest/inexplicit reporting. To advance their careers, researchers are also “gaming the system” by manipulating citation metrics and publishing in predatory and/or pay-to-publish journals that lack robust peer-review. The consequences of QRPs in the discipline may be profound: from increasing the false positivity rate to eroding public trust in the very institutions tasked with informing public-health policy. But what are the incentives underpinning misconduct and QRPs? And what are the solutions? This narrative review is a consciousness raiser that explores i) the manifestations of QRPs in kinesiology; ii) the excessive publication pressures, funding pressures, and performance incentives that are likely responsible; and iii) possible solutions for reform.
Metrics
References
Abt, G., Boreham, C., Davison, G., Jackson, R., Nevill, A., Wallace, E., & Williams, M. (2020). Power, precision, and sample size estimation in sport and exercise science research. Journal of Sports Sciences, 38(17), 1933–1935. https://doi.org/10.1080/02640414.2020.1776002
Abt, G., Jobson, S., Morin, J.-B., Passfield, L., Sampaio, J., Sunderland, C., & Twist, C. (2022). Raising the bar in sports performance research. Journal of Sports Sciences, 40(2), 125–129. https://doi.org/10.1080/02640414.2021.2024334
Altman, D. G., Gore, S. M., Gardner, M. J., & Pocock, S. J. (1983). Statistical guidelines for contributors to medical journals. British Medical Journal (Clinical Research Ed.), 286(6376), 1489–1493.
American Psychological Association. (2020). Publication manual of the American Psychological Association 2020: The official guide to APA style (7th ed.).
Andersen, M. B., McCullagh, P., & Wilson, G. J. (2007). But what do the numbers really tell us? Arbitrary metrics and effect size reporting in sport psychology research. Journal of Sport & Exercise Psychology, 29(5), 664–672. https://doi.org/10.1123/jsep.29.5.664
Andrade, C. (2015). The primary outcome measure and its importance in clinical trials. The Journal of Clinical Psychiatry, 76(10), e1320-1323. https://doi.org/10.4088/JCP.15f10377
Aslam, A., Imanullah, S., Asim, M., & El-Menyar, A. (2013). Registration of clinical trials: Is it really needed? North American Journal of Medical Sciences, 5(12), 713–715. https://doi.org/10.4103/1947-2714.123266
Atkinson, G., & Batterham, A. M. (2015). True and false interindividual differences in the physiological response to an intervention. Experimental Physiology, 100(6), 577–588. https://doi.org/10.1113/EP085070
Atkinson, G., Loenneke, J. P., Fahs, C. A., Abe, T., & Rossow, L. M. (2015). Individual differences in the exercise-mediated blood pressure response: Regression to the mean in disguise? Clinical Physiology and Functional Imaging, 35(6), 490–491. https://doi.org/10.1111/cpf.12211
Atkinson, G., & Nevill, A. M. (2001). Selected issues in the design and analysis of sport performance research. Journal of Sports Sciences, 19(10), 811–827. https://doi.org/10.1080/026404101317015447
Atkinson, G., Williamson, P., & Batterham, A. M. (2019). Issues in the determination of “responders” and “non-responders” in physiological research. Experimental Physiology, 104(8), 1215–1225. https://doi.org/10.1113/EP087712
Bachynski, K. E., & Smoliga, J. M. (2021). Pseudomedicine for sports concussions in the USA. The Lancet. Neurology, 20(10), 791–792. https://doi.org/10.1016/S1474-4422(19)30250-9
Barlow, P., Serôdio, P., Ruskin, G., McKee, M., & Stuckler, D. (2018). Science organisations and Coca-Cola’s “war” with the public health community: Insights from an internal industry document. Journal of Epidemiology and Community Health, 72(9), 761–763. https://doi.org/10.1136/jech-2017-210375
Baugh, F. (2002). Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably. Educational and Psychological Measurement, 62(2), 254–263. https://doi.org/10.1177/0013164402062002004
Begley, C. G., & Ioannidis, J. P. A. (2015). Reproducibility in science: Improving the standard for basic and preclinical research. Circulation Research, 116(1), 116–126. https://doi.org/10.1161/CIRCRESAHA.114.303819
Bernard, C. (2019). Changing the way we report, interpret, and discuss our results to rebuild trust in our research. ENeuro, 6(4), ENEURO.0259-19.2019. https://doi.org/10.1523/ENEURO.0259-19.2019
Bernards, J. R., Sato, K., Haff, G. G., & Bazyler, C. D. (2017). Current research and statistical practices in sport science and a need for change. Sports, 5(4), Article 4. https://doi.org/10.3390/sports5040087
Bes-Rastrollo, M., Schulze, M. B., Ruiz-Canela, M., & Martinez-Gonzalez, M. A. (2013). Financial conflicts of interest and reporting bias regarding the association between sugar-sweetened beverages and weight gain: A systematic review of systematic reviews. PLoS Medicine, 10(12), e1001578; dicsussion e1001578. https://doi.org/10.1371/journal.pmed.1001578
Biagioli, M., & Lippman, A. (2020). Gaming the metrics: Misconduct and manipulation in academic research. MIT Press. https://doi.org/10.7551/mitpress/11087.001.0001
Bishop, D. (2008). An applied research model for the sport sciences. Sports Medicine (Auckland, N.Z.), 38(3), 253–263. https://doi.org/10.2165/00007256-200838030-00005
Bleakley, C., & MacAuley, D. (2002). The quality of research in sports journals. British Journal of Sports Medicine, 36(2), 124–125. https://doi.org/10.1136/bjsm.36.2.124
Blumenthal, J. A., Smith, P. J., Mabe, S., Hinderliter, A., Lin, P.-H., Liao, L., Welsh-Bohmer, K. A., Browndyke, J. N., Kraus, W. E., Doraiswamy, P. M., Burke, J. R., & Sherwood, A. (2019). Lifestyle and neurocognition in older adults with cognitive impairments: A randomized trial. Neurology, 92(3), e212–e223. https://doi.org/10.1212/WNL.0000000000006784
Bonafiglia, J. T., Islam, H., Preobrazenski, N., & Gurd, B. J. (2022). Risk of bias and reporting practices in studies comparing VO2max responses to sprint interval vs. continuous training: A systematic review and meta-analysis. Journal of Sport and Health Science, 11(5), 552–566. https://doi.org/10.1016/j.jshs.2021.03.005
Brainard, J. (2020). Articles in “predatory” journals receive few or no citations. Science (New York, N.Y.), 367(6474), 129. https://doi.org/10.1126/science.367.6474.129
Brischoux, F., & Angelier, F. (2015). Academia’s never-ending selection for productivity. Scientometrics, 103(1), 333–336. https://doi.org/10.1007/s11192-015-1534-5
Büttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020). Are questionable research practices facilitating new discoveries in sport and exercise medicine? The proportion of supported hypotheses is implausibly high. British Journal of Sports Medicine, 54(22), 1365–1371. https://doi.org/10.1136/bjsports-2019-101863
Caldwell, A. R., Vigotsky, A. D., Tenan, M. S., Radel, R., Mellor, D. T., Kreutzer, A., Lahart, I. M., Mills, J. P., Boisgontier, M. P., & Consortium for Transparency in Exercise Science (COTES) Collaborators. (2020). Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices. Sports Medicine (Auckland, N.Z.), 50(3), 449–459. https://doi.org/10.1007/s40279-019-01227-1
Callaway, E. (2011). Fraud investigation rocks Danish university. Nature. https://doi.org/10.1038/news.2011.703
Chahal, J., Tomescu, S. S., Ravi, B., Bach, B. R., Ogilvie-Harris, D., Mohamed, N. N., & Gandhi, R. (2012). Publication of sports medicine-related randomized controlled trials registered in ClinicalTrials.gov. The American Journal of Sports Medicine, 40(9), 1970–1977. https://doi.org/10.1177/0363546512448363
Chan, A.-W., Hróbjartsson, A., Jørgensen, K. J., Gøtzsche, P. C., & Altman, D. G. (2008). Discrepancies in sample size calculations and data analyses reported in randomised trials: Comparison of publications with protocols. BMJ (Clinical Research Ed.), 337, a2299. https://doi.org/10.1136/bmj.a2299
Charles, P., Giraudeau, B., Dechartres, A., Baron, G., & Ravaud, P. (2009). Reporting of sample size calculation in randomised controlled trials: Review. BMJ (Clinical Research Ed.), 338, b1732. https://doi.org/10.1136/bmj.b1732
Charter, R. A. (1997). Effect on measurement error on tests of statistical significance. Journal of Clinical and Experimental Neuropsychology, 19(3), 458–462. https://doi.org/10.1080/01688639708403872
Christensen, J. E., & Christensen, C. E. (1977). Statistical power analysis of health, physical education, and recreation research. Research Quarterly, 48(1), 204–208. https://doi.org/10.1080/10671315.1977.10762173
Colloca, L. (2019). The Placebo Effect in Pain Therapies. Annual Review of Pharmacology and Toxicology, 59, 191–211. https://doi.org/10.1146/annurev-pharmtox-010818-021542
Coursol, A., & Wagner, E. E. (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice, 17(2), 136–137. https://doi.org/10.1037/0735-7028.17.2.136
Cumming, G. (2008). Replication and p Intervals: P Values Predict the Future Only Vaguely, but Confidence Intervals Do Much Better. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 3(4), 286–300. https://doi.org/10.1111/j.1745-6924.2008.00079.x
Cumming, G., Fidler, F., Kalinowski, P., & Lai, J. (2012). The statistical recommendations of the American psychological association publication manual: Effect sizes, confidence intervals, and meta‐analysis. Australian Journal of Psychology, 64, 138–146. https://doi.org/10.1111/j.1742-9536.2011.00037.x
Curran-Everett, D. (2016). Explorations in statistics: Statistical facets of reproducibility. Advances in Physiology Education, 40(2), 248–252. https://doi.org/10.1152/advan.00042.2016
Curran-Everett, D., & Benos, D. J. (2007). Guidelines for reporting statistics in journals published by the American Physiological Society: The sequel. Advances in Physiology Education, 31(4), 295–298. https://doi.org/10.1152/advan.00022.2007
Dahlberg, J. E., & Mahler, C. C. (2006). The poehlman case: Running away from the truth. Science and Engineering Ethics, 12(1), 157–173. https://doi.org/10.1007/s11948-006-0016-9
Dalton, R. (2005). Obesity expert owns up to million-dollar crime. Nature, 434(7032), Article 7032. https://doi.org/10.1038/434424a
Del Coso, J., Lara, B., Ruiz-Moreno, C., & Salinero, J. J. (2019). Challenging the Myth of Non-Response to the Ergogenic Effects of Caffeine Ingestion on Exercise Performance. Nutrients, 11(4), E732. https://doi.org/10.3390/nu11040732
Desharnais, R., Jobin, J., Côté, C., Lévesque, L., & Godin, G. (1993). Aerobic exercise and the placebo effect: A controlled study. Psychosomatic Medicine, 55(2), 149–154. https://doi.org/10.1097/00006842-199303000-00003
Diels, J., Cunha, M., Manaia, C., Sabugosa-Madeira, B., & Silva, M. (2011). Association of financial or professional conflict of interest to research outcomes on health risks or nutritional assessment studies of genetically modified products. Food Policy, 36(2), 197–203. https://doi.org/10.1016/j.foodpol.2010.11.016
Dinis-Oliveira, R. J., & Magalhães, T. (2016). The Inherent Drawbacks of the Pressure to Publish in Health Sciences: Good or Bad Science. F1000Research, 4, 419. https://doi.org/10.12688/f1000research.6809.2
Diong, J., Butler, A. A., Gandevia, S. C., & Héroux, M. E. (2018). Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PloS One, 13(8), e0202121. https://doi.org/10.1371/journal.pone.0202121
DuBois, J. M., Dueker, J. M., Anderson, E. E., & Campbell, J. (2008). The development and assessment of an NIH-funded research ethics training program. Academic Medicine: Journal of the Association of American Medical Colleges, 83(6), 596–603. https://doi.org/10.1097/ACM.0b013e3181723095
Dumas-Mallet, E., Button, K. S., Boraud, T., Gonon, F., & Munafò, M. R. (2017). Low statistical power in biomedical science: A review of three human research domains. Royal Society Open Science, 4(2), 160254. https://doi.org/10.1098/rsos.160254
Earnest, C. P., Roberts, B. M., Harnish, C. R., Kutz, J. L., Cholewa, J. M., & Johannsen, N. M. (2018). Reporting Characteristics in Sports Nutrition. Sports, 6(4), Article 4. https://doi.org/10.3390/sports6040139
Edwards, M. A., & Roy, S. (2016). Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. Environmental Engineering Science, 34(1), 51–61. https://doi.org/10.1089/ees.2016.0223
Eisenmann, J. (2017). Translational Gap between Laboratory and Playing Field: New Era to Solve Old Problems in Sports Science. Translational Journal of the American College of Sports Medicine, 2(8), 37–43. https://doi.org/10.1249/TJX.0000000000000032
Ekkekakis, P., & Tiller, N. B. (2022). Extraordinary Claims in the Literature on High-Intensity Interval Training: II. Are the Extraordinary Claims Supported by Extraordinary Evidence? Kinesiology Review, 1(aop), 1–14. https://doi.org/10.1123/kr.2022-0003
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. ELife, 10, e71601. https://doi.org/10.7554/eLife.71601
Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PloS One, 4(5), e5738. https://doi.org/10.1371/journal.pone.0005738
Fanelli, D. (2010). “Positive” Results Increase Down the Hierarchy of the Sciences. PLOS ONE, 5(4), e10068. https://doi.org/10.1371/journal.pone.0010068
Fanelli, D., Costas, R., & Larivière, V. (2015). Misconduct Policies, Academic Culture and Career Stage, Not Gender or Pressures to Publish, Affect Scientific Integrity. PloS One, 10(6), e0127556. https://doi.org/10.1371/journal.pone.0127556
Fisher, R. A. (1926). The arrangement of field experiments. Journal of the Ministry of Agriculture, 33, 503–515. https://doi.org/10.23637/rothamsted.8v61q
Flanagin, A., Fontanarosa, P. B., & Bauchner, H. (2020). Preprints Involving Medical Research-Do the Benefits Outweigh the Challenges? JAMA, 324(18), 1840–1843. https://doi.org/10.1001/jama.2020.20674
Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505. https://doi.org/10.1126/science.1255484
Freemantle, N. (2001). Interpreting the results of secondary end points and subgroup analyses in clinical trials: Should we lock the crazy aunt in the attic? BMJ, 322(7292), 989–991. https://doi.org/10.1136/bmj.322.7292.989
Gandevia, S. (2021). Publications, replication and statistics in physiology plus two neglected curves. The Journal of Physiology, 599(6), 1719–1721. https://doi.org/10.1113/JP281360
Gardner, M. J., Altman, D. G., Jones, D. R., & Machin, D. (1983). Is the statistical assessment of papers submitted to the “British Medical Journal” effective? Br Med J (Clin Res Ed), 286(6376), 1485–1488. https://doi.org/10.1136/bmj.286.6376.1485
Garvican, L. A., Pottgiesser, T., Martin, D. T., Schumacher, Y. O., Barras, M., & Gore, C. J. (2011). The contribution of haemoglobin mass to increases in cycling performance induced by simulated LHTL. European Journal of Applied Physiology, 111(6), 1089–1101. https://doi.org/10.1007/s00421-010-1732-z
Gaspar, D. E. P., & Esteves, M. D. L. (2021). Awareness of the Misconduct in Sports Science Research. Annals of Applied Sport Science, 9(3), 0–0. https://doi.org/10.52547/aassjournal.934
Gasparyan, A. Y., Ayvazyan, L., Akazhanov, N. A., & Kitas, G. D. (2014). Self-correction in biomedical publications and the scientific impact. Croatian Medical Journal, 55(1), 61–72. https://doi.org/10.3325/cmj.2014.55.61
Gerrits, R. G., Jansen, T., Mulyanto, J., Berg, M. J. van den, Klazinga, N. S., & Kringos, D. S. (2019). Occurrence and nature of questionable research practices in the reporting of messages and conclusions in international scientific Health Services Research publications: A structured assessment of publications authored by researchers in the Netherlands. BMJ Open, 9(5), e027903. https://doi.org/10.1136/bmjopen-2018-027903
Gibbs, K. D., & Griffin, K. A. (2013). What Do I Want to Be with My PhD? The Roles of Personal Values and Structural Dynamics in Shaping the Career Interests of Recent Biomedical Science PhD Graduates. CBE Life Sciences Education, 12(4), 711–723. https://doi.org/10.1187/cbe.13-02-0021
Gibbs, N. M., & Gibbs, S. V. (2015). Misuse of “trend” to describe “almost significant” differences in anaesthesia research. British Journal of Anaesthesia, 115(3), 337–339. https://doi.org/10.1093/bja/aev149
Goodman, S. N. (1992). A comment on replication, P-values and evidence. Statistics in Medicine, 11(7), 875–879. https://doi.org/10.1002/sim.4780110705
Gopalakrishna, G., Ter Riet, G., Vink, G., Stoop, I., Wicherts, J. M., & Bouter, L. M. (2022). Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands. PloS One, 17(2), e0263023. https://doi.org/10.1371/journal.pone.0263023
Groenwold, R. H. H., & Dekkers, O. M. (2020). Measurement error in clinical research, yes it matters. European Journal of Endocrinology, 183(3), E3–E5. https://doi.org/10.1530/EJE-20-0550
Halperin, I., Vigotsky, A. D., Foster, C., & Pyne, D. B. (2018). Strengthening the Practice of Exercise and Sport-Science Research. International Journal of Sports Physiology and Performance, 13(2), 127–134. https://doi.org/10.1123/ijspp.2017-0322
Harris, J. D., Cvetanovich, G., Erickson, B. J., Abrams, G. D., Chahal, J., Gupta, A. K., McCormick, F. M., & Bach, B. R. (2014). Current status of evidence-based sports medicine. Arthroscopy: The Journal of Arthroscopic & Related Surgery: Official Publication of the Arthroscopy Association of North America and the International Arthroscopy Association, 30(3), 362–371. https://doi.org/10.1016/j.arthro.2013.11.015
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a
Higginson, A. D., & Munafò, M. R. (2016). Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions. PLOS Biology, 14(11), e2000995. https://doi.org/10.1371/journal.pbio.2000995
Hopkins, W. G., Marshall, S. W., Batterham, A. M., & Hanin, J. (2009). Progressive Statistics for Studies in Sports Medicine and Exercise Science. Medicine & Science in Sports & Exercise, 41(1), 3–12. https://doi.org/10.1249/MSS.0b013e31818cb278
How to spot dodgy academic journals. (2020). The Economist. https://www.economist.com/graphic-detail/2020/05/30/how-to-spot-dodgy-academic-journals
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
Ioannidis, J. P. A., Boyack, K. W., & Baas, J. (2020). Updated science-wide author databases of standardized citation indicators. PLoS Biology, 18(10), e3000918. https://doi.org/10.1371/journal.pbio.3000918
Ioannidis, J. P. A., Ntzani, E. E., Trikalinos, T. A., & Contopoulos-Ioannidis, D. G. (2001). Replication validity of genetic association studies. Nature Genetics, 29(3), Article 3. https://doi.org/10.1038/ng749
Islam, H., & Gurd, B. J. (2020). Exercise response variability: Random error or true differences in exercise response? Experimental Physiology, 105(12), 2022–2024. https://doi.org/10.1113/EP089015
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953
Jones, S. R., Carley, S., & Harrison, M. (2003). An introduction to power and sample size estimation. Emergency Medicine Journal, 20(5), 453–458. https://doi.org/10.1136/emj.20.5.453
Kardeş, S., Levack, W., Özkuk, K., Atmaca Aydın, E., & Seringeç Karabulut, S. (2020). Retractions in Rehabilitation and Sport Sciences Journals: A Systematic Review. Archives of Physical Medicine and Rehabilitation, 101(11), 1980–1990. https://doi.org/10.1016/j.apmr.2020.03.010
Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4
Khatra, O., Shadgan, A., Taunton, J., Pakravan, A., & Shadgan, B. (2021). A bibliometric analysis of the top cited articles in sports and exercise medicine. Orthopaedic Journal of Sports Medicine, 9(1), 2325967120969902. https://doi.org/10.1177/2325967120969902
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biology, 14(5), e1002456. https://doi.org/10.1371/journal.pbio.1002456
King, E. G., Oransky, I., Sachs, T. E., Farber, A., Flynn, D. B., Abritis, A., Kalish, J. A., & Siracuse, J. J. (2018). Analysis of retracted articles in the surgical literature. American Journal of Surgery, 216(5), 851–855. https://doi.org/10.1016/j.amjsurg.2017.11.033
Kintisch, E. (2006). Poehlman Sentenced to 1 Year of Prison. Science.Org. https://www.science.org/content/article/poehlman-sentenced-1-year-prison
Kiss, A., Temesi, Á., Tompa, O., Lakner, Z., & Soós, S. (2021). Structure and trends of international sport nutrition research between 2000 and 2018: Bibliometric mapping of sport nutrition science. Journal of the International Society of Sports Nutrition, 18(1), 12. https://doi.org/10.1186/s12970-021-00409-5
Klimentidis, Y. C., Raichlen, D. A., Bea, J., Garcia, D. O., Wineinger, N. E., Mandarino, L. J., Alexander, G. E., Chen, Z., & Going, S. B. (2018). Genome-wide association study of habitual physical activity in over 377,000 UK Biobank participants identifies multiple variants including CADM2 and APOE. International Journal of Obesity (2005), 42(6), 1161–1176. https://doi.org/10.1038/s41366-018-0120-3
Knudson, D. (2009). Significant and meaningful effects in sports biomechanics research. Sports Biomechanics, 8(1), 96–104. https://doi.org/10.1080/14763140802629966
Knudson, D. (2012). Twenty-year trends of authorship and sampling in applied biomechanics research. Perceptual and Motor Skills, 114(1), 16–20. https://doi.org/10.2466/11.PMS.114.1.16-20
Knudson, D. (2017a). Twenty Years of Authorship, Sampling, and References in Kinesiology Research Reports. International Journal of Kinesiology in Higher Education, 1(2), 44–52. https://doi.org/10.1080/24711616.2017.1282760
Knudson, D. (2017b). Confidence crisis of results in biomechanics research. Sports Biomechanics, 16(4), 425–433. https://doi.org/10.1080/14763141.2016.1246603
Knudson, D. (2019). Judicious Use of Bibliometrics to Supplement Peer Evaluations of Research in Kinesiology in: Kinesiology Review Volume 8 Issue 2 (). Kinesiology Review, 8(2), 100–109. https://doi.org/doi.org/10.1123/kr.2017-0046
Kondro, W. (2005). Université de Montréal in the dark about fraud. CMAJ, 172(10), 1278-1278-a. https://doi.org/10.1503/cmaj.050455
Krans, B. (2022). Lawsuit: Coca-Cola Fake Ads About Obesity. Healthline.Com. https://www.healthline.com/health/coca-cola-false-advertising-unhealthy-drinks
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863
Landhuis, E. (2016). Scientific literature: Information overload. Nature, 535(7612), Article 7612. https://doi.org/10.1038/nj7612-457a
Lesser, L. I., Ebbeling, C. B., Goozner, M., Wypij, D., & Ludwig, D. S. (2007). Relationship between funding source and conclusion among nutrition-related scientific articles. PLoS Medicine, 4(1), e5. https://doi.org/10.1371/journal.pmed.0040005
Lohse, K. R., Sainani, K. L., Taylor, J. A., Butson, M. L., Knight, E. J., & Vickers, A. J. (2020). Systematic review of the use of “magnitude-based inference” in sports science and medicine. PLOS ONE, 15(6), e0235318. https://doi.org/10.1371/journal.pone.0235318
Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. Science (New York, N.Y.), 355(6325), 584–585. https://doi.org/10.1126/science.aal3618
Lucas, M. (2015). Conflicts of interest in nutritional sciences: The forgotten bias in meta-analysis. World Journal of Methodology, 5(4), 175–178. https://doi.org/10.5662/wjm.v5.i4.175
Maggio, L., Dong, T., Driessen, E., & Artino, A. (2019). Factors associated with scientific misconduct and questionable research practices in health professions education. Perspectives on Medical Education, 8(2), 74–82. https://doi.org/10.1007/s40037-019-0501-x
Mallapaty, S. (2020). China bans cash rewards for publishing papers. Nature, 579(7797), 18. https://doi.org/10.1038/d41586-020-00574-8
Marticorena, F. M., Carvalho, A., Oliveira, L. F. D., Dolan, E., Gualano, B., Swinton, P., & Saunders, B. (2021). Nonplacebo Controls to Determine the Magnitude of Ergogenic Interventions: A Systematic Review and Meta-analysis. Medicine and Science in Sports and Exercise, 53(8), 1766–1777. https://doi.org/10.1249/MSS.0000000000002635
Martin, J. J. (2020). Grants: The Good, the Bad, the Ugly, and the Puzzling. Kinesiology Review, 10(1), 18–28. https://doi.org/10.1123/kr.2020-0013
Mattioni Maturana, F., Martus, P., Zipfel, S., & NIEß, A. M. (2021). Effectiveness of HIIE versus MICT in Improving Cardiometabolic Risk Factors in Health and Disease: A Meta-analysis. Medicine and Science in Sports and Exercise, 53(3), 559–573. https://doi.org/10.1249/MSS.0000000000002506
Montero, D., & Lundby, C. (2017). Refuting the myth of non‐response to exercise training: ‘Non‐responders’ do respond to higher dose of training. The Journal of Physiology, 595(11), 3377–3387. https://doi.org/10.1113/JP273480
Moylan, E. C., & Kowalczuk, M. K. (2016). Why articles are retracted: A retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open, 6(11), e012047. https://doi.org/10.1136/bmjopen-2016-012047
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., du Sert, N. P., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021. https://doi.org/10.1038/s41562-016-0021
Murphy, J., Mesquida, C., Caldwell, A. R., Earp, B. D., & Warne, J. P. (2023). Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science. Sports Medicine (Auckland, N.Z.). https://doi.org/10.1007/s40279-022-01749-1
Nagele, P. (2003). Misuse of standard error of the mean (sem) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. BJA: British Journal of Anaesthesia, 90(4), 514–516. https://doi.org/10.1093/bja/aeg087
Nosek, B. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
Nosek, B. A., & Bar-Anan, Y. (2012). Scientific Utopia: I. Opening scientific communication. ArXiv:1205.1055 [Physics]. http://arxiv.org/abs/1205.1055
Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(6), 615–631. https://doi.org/10.1177/1745691612459058
Open Science Collaboration. (2015). PSYCHOLOGY. Estimating the reproducibility of psychological science. Science (New York, N.Y.), 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
Pabst, S., Brand, M., & Wolf, O. T. (2013). Stress effects on framed decisions: There are differences for gains and losses. Frontiers in Behavioral Neuroscience, 7, 142. https://doi.org/10.3389/fnbeh.2013.00142
Padilla, J., Leary, E., & Limberg, J. K. (2021). Identifying responders versus non-responders: Incorporation of controls is required for sound statistical inference. Experimental Physiology, 106(2), 375–376. https://doi.org/10.1113/EP089142
Prom-Wormley, E., Adkins, A., Waldman, I. D., & Dick, D. (2017). Critical Issues in Genetic Association Studies. In Psychological Science Under Scrutiny (pp. 221–249). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781119095910.ch12
Quan, W., Chen, B., & Shu, F. (2017). Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016). Aslib Journal of Information Management, 69(5), 486–502. https://doi.org/10.1108/AJIM-01-2017-0014
Robertson, P. S. (2020). Man & machine: Adaptive tools for the contemporary performance analyst. Journal of Sports Sciences, 38(18), 2118–2126. https://doi.org/10.1080/02640414.2020.1774143
Sabroe, I., Dockrell, D. H., Vogel, S. N., Renshaw, S. A., Whyte, M. K. B., & Dower, S. K. (2007). Identifying and hurdling obstacles to translational research. Nature Reviews. Immunology, 7(1), 77–82. https://doi.org/10.1038/nri1999
Sainani, K., & Chamari, K. (2022). Wish List for Improving the Quality of Statistics in Sport Science. International Journal of Sports Physiology and Performance, 17(5), 673–674. https://doi.org/10.1123/ijspp.2022-0023
Sainani, K. L., Borg, D. N., Caldwell, A. R., Butson, M. L., Tenan, M. S., Vickers, A. J., Vigotsky, A. D., Warmenhoven, J., Nguyen, R., Lohse, K. R., Knight, E. J., & Bargary, N. (2021). Call to increase statistical collaboration in sports science, sport and exercise medicine and sports physiotherapy. British Journal of Sports Medicine, 55(2), 118–122. https://doi.org/10.1136/bjsports-2020-102607
Sarabipour, S., Debat, H. J., Emmott, E., Burgess, S. J., Schwessinger, B., & Hensel, Z. (2019). On the value of preprints: An early career researcher perspective. PLoS Biology, 17(2), e3000151. https://doi.org/10.1371/journal.pbio.3000151
Scheel, A. M., Schijen, M., & Lakens, D. (2020). An excess of positive results: Comparing the standard Psychology literature with Registered Reports. PsyArXiv. https://doi.org/10.31234/osf.io/p6e9c
Schmid, S. L. (2017). Five years post-DORA: Promoting best practices for research assessment. Molecular Biology of the Cell, 28(22), 2941–2944. https://doi.org/10.1091/mbc.E17-08-0534
Schulz, K. F., Altman, D. G., Moher, D., & CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. BMC Medicine, 8, 18. https://doi.org/10.1186/1741-7015-8-18
Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. L. (2022). Reporting and transparent research practices in sports medicine and orthopaedic clinical trials: A meta-research study. BMJ Open, 12(8), e059347. https://doi.org/10.1136/bmjopen-2021-059347
Serodio, P., Ruskin, G., McKee, M., & Stuckler, D. (2020). Evaluating Coca-Cola’s attempts to influence public health ‘in their own words’: Analysis of Coca-Cola emails with public health academics leading the Global Energy Balance Network. Public Health Nutrition, 23, 1–7. https://doi.org/10.1017/S1368980020002098
Serra-Garcia, M., & Gneezy, U. (2021). Nonreplicable publications are cited more than replicable ones. Science Advances, 7(21), eabd1705. https://doi.org/10.1126/sciadv.abd1705
Shen, C., & Björk, B.-C. (2015). “Predatory” open access: A longitudinal study of article volumes and market characteristics. BMC Medicine, 13, 230. https://doi.org/10.1186/s12916-015-0469-2
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., … Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/10.1177/2515245917747646
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
Smaldino, P. E., & McElreath, R. (2016). The Natural Selection of Bad Science. Royal Society Open Science, 3(9), 160384. https://doi.org/10.1098/rsos.160384
Smoliga, J. M. (2020). Interpreting Biomarker Data After Concussion and Repeated Subconcussive Head Impacts: Challenges in Evaluating Brain Protection. JAMA Neurology, 77(12), 1477–1478. https://doi.org/10.1001/jamaneurol.2020.3467
Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021). Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour, 5(8), Article 8. https://doi.org/10.1038/s41562-021-01142-4
Sox, H. C., & Rennie, D. (2006). Research Misconduct, Retraction, and Cleansing the Medical Literature: Lessons from the Poehlman Case. Annals of Internal Medicine, 144(8), 609–613. https://doi.org/10.7326/0003-4819-144-8-200604180-00123
Spence, J. C., & Blanchard, C. (2001). Publication bias in sport and exercise psychology: The games we play. International Journal of Sport Psychology, 32(4), 386–399.
Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018). What meta-analyses reveal about the replicability of psychological research. Psychological Bulletin, 144(12), 1325–1346. https://doi.org/10.1037/bul0000169
Statista. (2020). Global sports nutrition & supplement market 2025. Statista. https://www.statista.com/statistics/450168/global-sports-nutrition-market/
Steen, R. G. (2011). Misinformation in the medical literature: What role do error and fraud play? Journal of Medical Ethics, 37(8), 498–503. https://doi.org/10.1136/jme.2010.041830
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995). Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa. The American Statistician, 49(1), 108–112. https://doi.org/10.2307/2684823
Stuckler, D., Ruskin, G., & McKee, M. (2018). Complexity and conflicts of interest statements: A case-study of emails exchanged between Coca-Cola and the principal investigators of the International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE). Journal of Public Health Policy, 39(1), 49–56. https://doi.org/10.1057/s41271-017-0095-7
Teo, Y. Y. (2008). Common statistical issues in genome-wide association studies: A review on power, data quality control, genotype calling and population structure. Current Opinion in Lipidology, 19(2), 133–143. https://doi.org/10.1097/MOL.0b013e3282f5dd77
The Society for Transparency, Openness, and Replication in Kinesiology. (n.d.). Reports in Sport and Exercise. Retrieved March 3, 2022, from https://storkjournals.org/index.php/rrik
Thomas, J. R., Salazar, W., & Landers, D. M. (1991). What is missing in p less than .05? Effect size. Research Quarterly for Exercise and Sport, 62(3), 344–348. https://doi.org/10.1080/02701367.1991.10608733
Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26–30. https://doi.org/10.2307/1176337
Tijdink, J. K., Verbeke, R., & Smulders, Y. M. (2014). Publication Pressure and Scientific Misconduct in Medical Scientists. Journal of Empirical Research on Human Research Ethics, 9(5), 64–71. https://doi.org/10.1177/1556264614552421
Tiller, N. B. (2020). The Skeptic’s Guide to Sports Science: Confronting Myths of the Health and Fitness Industry. Routledge.
Tiller, N. B., Sullivan, J. P., & Ekkekakis, P. (2022). Baseless Claims and Pseudoscience in Health and Wellness: A Call to Action for the Sports, Exercise, and Nutrition-Science Community. Sports Medicine (Auckland, N.Z.). https://doi.org/10.1007/s40279-022-01702-2
Twomey, R., Yingling, V., Warne, J., Schneider, C., McCrum, C., Atkins, W., Murphy, J., Medina, C. R., Harlley, S., & Caldwell, A. (2021). The Nature of Our Literature: A Registered Report on the Positive Result Rate and Reporting Practices in Kinesiology. Communications in Kinesiology, 1(3), Article 3. https://doi.org/10.51224/cik.v1i3.43
Vagenas, G., Palaiothodorou, D., & Knudson, D. (2018). Thirty-year Trends of Study Design and Statistics in Applied Sports and Exercise Biomechanics Research. International Journal of Exercise Science, 11(1), 239–259.
van de Schoot, R., Winter, S. D., Griffioen, E., Grimmelikhuijsen, S., Arts, I., Veen, D., Grandfield, E. M., & Tummers, L. G. (2021). The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum. Frontiers in Psychology, 12, 621547. https://doi.org/10.3389/fpsyg.2021.621547
Van Noorden, R. (2010). Metrics: A profusion of measures. Nature, 465(7300), 864–866. https://doi.org/10.1038/465864a
Van Noorden, R., & Singh Chawla, D. (2019). Hundreds of extreme self-citing scientists revealed in new database. Nature, 572(7771), 578–579. https://doi.org/10.1038/d41586-019-02479-7
Vankov, I., Bowers, J., & Munafò, M. R. (2014). Article Commentary: On the Persistence of Low Power in Psychological Science. Quarterly Journal of Experimental Psychology, 67(5), 1037–1040. https://doi.org/10.1080/17470218.2014.885986
Viken, H., Reitlo, L. S., Zisko, N., Nauman, J., Aspvik, N. P., Ingebrigtsen, J. E., Wisløff, U., & Stensvold, D. (2019). Predictors of Dropout in Exercise Trials in Older Adults: The Generation 100 Study. Medicine & Science in Sports & Exercise, 51(1), 49–55. https://doi.org/10.1249/MSS.0000000000001742
Walters, J. (2015, August 11). Nutrition experts alarmed by nonprofit downplaying role of junk food in obesity. The Guardian. https://www.theguardian.com/society/2015/aug/11/obesity-junk-food-exercise-global-energy-balance-network-coca-cola
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108
Watanabe, R. M. (2011). Statistical issues in gene association studies. Methods in Molecular Biology (Clifton, N.J.), 700, 17–36. https://doi.org/10.1007/978-1-61737-954-3_2
Wiedermann, C. J. (2016). Ethical publishing in intensive care medicine: A narrative review. World Journal of Critical Care Medicine, 5(3), 171–179. https://doi.org/10.5492/wjccm.v5.i3.171
Williams, C. J., Li, Z., Harvey, N., Lea, R. A., Gurd, B. J., Bonafiglia, J. T., Papadimitriou, I., Jacques, M., Croci, I., Stensvold, D., Wisloff, U., Taylor, J. L., Gajanand, T., Cox, E. R., Ramos, J. S., Fassett, R. G., Little, J. P., Francois, M. E., Hearon, C. M., … Coombes, J. S. (2021). Genome wide association study of response to interval and continuous exercise training: The Predict-HIIT study. Journal of Biomedical Science, 28(1), 37. https://doi.org/10.1186/s12929-021-00733-7
Wood, J., Freemantle, N., King, M., & Nazareth, I. (2014). Trap of trends to statistical significance: Likelihood of near significant P value becoming more significant with extra data. BMJ, 348, g2215. https://doi.org/10.1136/bmj.g2215
Yong, E. (2012). Nobel laureate challenges psychologists to clean up their act. Nature. https://doi.org/10.1038/nature.2012.11535
Downloads
Posted
Categories
License
Copyright (c) 2023 Nicholas Tiller, Panteleimon Ekkekakis
This work is licensed under a Creative Commons Attribution 4.0 International License.
