Preprint has been published in a journal as an article
Preprint / Version 1

Overcoming the “ostrich effect”

A narrative review on the incentives and consequences of questionable research practices in kinesiology

##article.authors##

  • Nicholas Tiller Harbor-UCLA
  • Panteleimon Ekkekakis Michigan State University

DOI:

https://doi.org/10.51224/SRXIV.273

Keywords:

ethics, exercise, health, methods, statistics, science

Abstract

Increasing transparency and openness in science is an ongoing endeavor, one that has stimulated self-reflection and reform in many fields. However, kinesiology and its related disciplines are among those exhibiting an “ostrich effect” and a reluctance to acknowledge their methodological shortcomings. Notwithstanding several high-profile cases of scientific misconduct, scholars in the field are frequently engaged in questionable research practices (QRPs), such as biased experimental designs, inappropriate statistics, and dishonest/inexplicit reporting. To advance their careers, researchers are also “gaming the system” by manipulating citation metrics and publishing in predatory and/or pay-to-publish journals that lack robust peer-review. The consequences of QRPs in the discipline may be profound: from increasing the false positivity rate to eroding public trust in the very institutions tasked with informing public-health policy. But what are the incentives underpinning misconduct and QRPs? And what are the solutions? This narrative review is a consciousness raiser that explores i) the manifestations of QRPs in kinesiology; ii) the excessive publication pressures, funding pressures, and performance incentives that are likely responsible; and iii) possible solutions for reform.

Metrics

Metrics Loading ...

References

Abt, G., Boreham, C., Davison, G., Jackson, R., Nevill, A., Wallace, E., & Williams, M. (2020). Power, precision, and sample size estimation in sport and exercise science research. Journal of Sports Sciences, 38(17), 1933–1935. https://doi.org/10.1080/02640414.2020.1776002

Abt, G., Jobson, S., Morin, J.-B., Passfield, L., Sampaio, J., Sunderland, C., & Twist, C. (2022). Raising the bar in sports performance research. Journal of Sports Sciences, 40(2), 125–129. https://doi.org/10.1080/02640414.2021.2024334

Altman, D. G., Gore, S. M., Gardner, M. J., & Pocock, S. J. (1983). Statistical guidelines for contributors to medical journals. British Medical Journal (Clinical Research Ed.), 286(6376), 1489–1493.

American Psychological Association. (2020). Publication manual of the American Psychological Association 2020: The official guide to APA style (7th ed.).

Andersen, M. B., McCullagh, P., & Wilson, G. J. (2007). But what do the numbers really tell us? Arbitrary metrics and effect size reporting in sport psychology research. Journal of Sport & Exercise Psychology, 29(5), 664–672. https://doi.org/10.1123/jsep.29.5.664

Andrade, C. (2015). The primary outcome measure and its importance in clinical trials. The Journal of Clinical Psychiatry, 76(10), e1320-1323. https://doi.org/10.4088/JCP.15f10377

Aslam, A., Imanullah, S., Asim, M., & El-Menyar, A. (2013). Registration of clinical trials: Is it really needed? North American Journal of Medical Sciences, 5(12), 713–715. https://doi.org/10.4103/1947-2714.123266

Atkinson, G., & Batterham, A. M. (2015). True and false interindividual differences in the physiological response to an intervention. Experimental Physiology, 100(6), 577–588. https://doi.org/10.1113/EP085070

Atkinson, G., Loenneke, J. P., Fahs, C. A., Abe, T., & Rossow, L. M. (2015). Individual differences in the exercise-mediated blood pressure response: Regression to the mean in disguise? Clinical Physiology and Functional Imaging, 35(6), 490–491. https://doi.org/10.1111/cpf.12211

Atkinson, G., & Nevill, A. M. (2001). Selected issues in the design and analysis of sport performance research. Journal of Sports Sciences, 19(10), 811–827. https://doi.org/10.1080/026404101317015447

Atkinson, G., Williamson, P., & Batterham, A. M. (2019). Issues in the determination of “responders” and “non-responders” in physiological research. Experimental Physiology, 104(8), 1215–1225. https://doi.org/10.1113/EP087712

Bachynski, K. E., & Smoliga, J. M. (2021). Pseudomedicine for sports concussions in the USA. The Lancet. Neurology, 20(10), 791–792. https://doi.org/10.1016/S1474-4422(19)30250-9

Barlow, P., Serôdio, P., Ruskin, G., McKee, M., & Stuckler, D. (2018). Science organisations and Coca-Cola’s “war” with the public health community: Insights from an internal industry document. Journal of Epidemiology and Community Health, 72(9), 761–763. https://doi.org/10.1136/jech-2017-210375

Baugh, F. (2002). Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably. Educational and Psychological Measurement, 62(2), 254–263. https://doi.org/10.1177/0013164402062002004

Begley, C. G., & Ioannidis, J. P. A. (2015). Reproducibility in science: Improving the standard for basic and preclinical research. Circulation Research, 116(1), 116–126. https://doi.org/10.1161/CIRCRESAHA.114.303819

Bernard, C. (2019). Changing the way we report, interpret, and discuss our results to rebuild trust in our research. ENeuro, 6(4), ENEURO.0259-19.2019. https://doi.org/10.1523/ENEURO.0259-19.2019

Bernards, J. R., Sato, K., Haff, G. G., & Bazyler, C. D. (2017). Current research and statistical practices in sport science and a need for change. Sports, 5(4), Article 4. https://doi.org/10.3390/sports5040087

Bes-Rastrollo, M., Schulze, M. B., Ruiz-Canela, M., & Martinez-Gonzalez, M. A. (2013). Financial conflicts of interest and reporting bias regarding the association between sugar-sweetened beverages and weight gain: A systematic review of systematic reviews. PLoS Medicine, 10(12), e1001578; dicsussion e1001578. https://doi.org/10.1371/journal.pmed.1001578

Biagioli, M., & Lippman, A. (2020). Gaming the metrics: Misconduct and manipulation in academic research. MIT Press. https://doi.org/10.7551/mitpress/11087.001.0001

Bishop, D. (2008). An applied research model for the sport sciences. Sports Medicine (Auckland, N.Z.), 38(3), 253–263. https://doi.org/10.2165/00007256-200838030-00005

Bleakley, C., & MacAuley, D. (2002). The quality of research in sports journals. British Journal of Sports Medicine, 36(2), 124–125. https://doi.org/10.1136/bjsm.36.2.124

Blumenthal, J. A., Smith, P. J., Mabe, S., Hinderliter, A., Lin, P.-H., Liao, L., Welsh-Bohmer, K. A., Browndyke, J. N., Kraus, W. E., Doraiswamy, P. M., Burke, J. R., & Sherwood, A. (2019). Lifestyle and neurocognition in older adults with cognitive impairments: A randomized trial. Neurology, 92(3), e212–e223. https://doi.org/10.1212/WNL.0000000000006784

Bonafiglia, J. T., Islam, H., Preobrazenski, N., & Gurd, B. J. (2022). Risk of bias and reporting practices in studies comparing VO2max responses to sprint interval vs. continuous training: A systematic review and meta-analysis. Journal of Sport and Health Science, 11(5), 552–566. https://doi.org/10.1016/j.jshs.2021.03.005

Brainard, J. (2020). Articles in “predatory” journals receive few or no citations. Science (New York, N.Y.), 367(6474), 129. https://doi.org/10.1126/science.367.6474.129

Brischoux, F., & Angelier, F. (2015). Academia’s never-ending selection for productivity. Scientometrics, 103(1), 333–336. https://doi.org/10.1007/s11192-015-1534-5

Büttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020). Are questionable research practices facilitating new discoveries in sport and exercise medicine? The proportion of supported hypotheses is implausibly high. British Journal of Sports Medicine, 54(22), 1365–1371. https://doi.org/10.1136/bjsports-2019-101863

Caldwell, A. R., Vigotsky, A. D., Tenan, M. S., Radel, R., Mellor, D. T., Kreutzer, A., Lahart, I. M., Mills, J. P., Boisgontier, M. P., & Consortium for Transparency in Exercise Science (COTES) Collaborators. (2020). Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices. Sports Medicine (Auckland, N.Z.), 50(3), 449–459. https://doi.org/10.1007/s40279-019-01227-1

Callaway, E. (2011). Fraud investigation rocks Danish university. Nature. https://doi.org/10.1038/news.2011.703

Chahal, J., Tomescu, S. S., Ravi, B., Bach, B. R., Ogilvie-Harris, D., Mohamed, N. N., & Gandhi, R. (2012). Publication of sports medicine-related randomized controlled trials registered in ClinicalTrials.gov. The American Journal of Sports Medicine, 40(9), 1970–1977. https://doi.org/10.1177/0363546512448363

Chan, A.-W., Hróbjartsson, A., Jørgensen, K. J., Gøtzsche, P. C., & Altman, D. G. (2008). Discrepancies in sample size calculations and data analyses reported in randomised trials: Comparison of publications with protocols. BMJ (Clinical Research Ed.), 337, a2299. https://doi.org/10.1136/bmj.a2299

Charles, P., Giraudeau, B., Dechartres, A., Baron, G., & Ravaud, P. (2009). Reporting of sample size calculation in randomised controlled trials: Review. BMJ (Clinical Research Ed.), 338, b1732. https://doi.org/10.1136/bmj.b1732

Charter, R. A. (1997). Effect on measurement error on tests of statistical significance. Journal of Clinical and Experimental Neuropsychology, 19(3), 458–462. https://doi.org/10.1080/01688639708403872

Christensen, J. E., & Christensen, C. E. (1977). Statistical power analysis of health, physical education, and recreation research. Research Quarterly, 48(1), 204–208. https://doi.org/10.1080/10671315.1977.10762173

Colloca, L. (2019). The Placebo Effect in Pain Therapies. Annual Review of Pharmacology and Toxicology, 59, 191–211. https://doi.org/10.1146/annurev-pharmtox-010818-021542

Coursol, A., & Wagner, E. E. (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice, 17(2), 136–137. https://doi.org/10.1037/0735-7028.17.2.136

Cumming, G. (2008). Replication and p Intervals: P Values Predict the Future Only Vaguely, but Confidence Intervals Do Much Better. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 3(4), 286–300. https://doi.org/10.1111/j.1745-6924.2008.00079.x

Cumming, G., Fidler, F., Kalinowski, P., & Lai, J. (2012). The statistical recommendations of the American psychological association publication manual: Effect sizes, confidence intervals, and meta‐analysis. Australian Journal of Psychology, 64, 138–146. https://doi.org/10.1111/j.1742-9536.2011.00037.x

Curran-Everett, D. (2016). Explorations in statistics: Statistical facets of reproducibility. Advances in Physiology Education, 40(2), 248–252. https://doi.org/10.1152/advan.00042.2016

Curran-Everett, D., & Benos, D. J. (2007). Guidelines for reporting statistics in journals published by the American Physiological Society: The sequel. Advances in Physiology Education, 31(4), 295–298. https://doi.org/10.1152/advan.00022.2007

Dahlberg, J. E., & Mahler, C. C. (2006). The poehlman case: Running away from the truth. Science and Engineering Ethics, 12(1), 157–173. https://doi.org/10.1007/s11948-006-0016-9

Dalton, R. (2005). Obesity expert owns up to million-dollar crime. Nature, 434(7032), Article 7032. https://doi.org/10.1038/434424a

Del Coso, J., Lara, B., Ruiz-Moreno, C., & Salinero, J. J. (2019). Challenging the Myth of Non-Response to the Ergogenic Effects of Caffeine Ingestion on Exercise Performance. Nutrients, 11(4), E732. https://doi.org/10.3390/nu11040732

Desharnais, R., Jobin, J., Côté, C., Lévesque, L., & Godin, G. (1993). Aerobic exercise and the placebo effect: A controlled study. Psychosomatic Medicine, 55(2), 149–154. https://doi.org/10.1097/00006842-199303000-00003

Diels, J., Cunha, M., Manaia, C., Sabugosa-Madeira, B., & Silva, M. (2011). Association of financial or professional conflict of interest to research outcomes on health risks or nutritional assessment studies of genetically modified products. Food Policy, 36(2), 197–203. https://doi.org/10.1016/j.foodpol.2010.11.016

Dinis-Oliveira, R. J., & Magalhães, T. (2016). The Inherent Drawbacks of the Pressure to Publish in Health Sciences: Good or Bad Science. F1000Research, 4, 419. https://doi.org/10.12688/f1000research.6809.2

Diong, J., Butler, A. A., Gandevia, S. C., & Héroux, M. E. (2018). Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PloS One, 13(8), e0202121. https://doi.org/10.1371/journal.pone.0202121

DuBois, J. M., Dueker, J. M., Anderson, E. E., & Campbell, J. (2008). The development and assessment of an NIH-funded research ethics training program. Academic Medicine: Journal of the Association of American Medical Colleges, 83(6), 596–603. https://doi.org/10.1097/ACM.0b013e3181723095

Dumas-Mallet, E., Button, K. S., Boraud, T., Gonon, F., & Munafò, M. R. (2017). Low statistical power in biomedical science: A review of three human research domains. Royal Society Open Science, 4(2), 160254. https://doi.org/10.1098/rsos.160254

Earnest, C. P., Roberts, B. M., Harnish, C. R., Kutz, J. L., Cholewa, J. M., & Johannsen, N. M. (2018). Reporting Characteristics in Sports Nutrition. Sports, 6(4), Article 4. https://doi.org/10.3390/sports6040139

Edwards, M. A., & Roy, S. (2016). Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. Environmental Engineering Science, 34(1), 51–61. https://doi.org/10.1089/ees.2016.0223

Eisenmann, J. (2017). Translational Gap between Laboratory and Playing Field: New Era to Solve Old Problems in Sports Science. Translational Journal of the American College of Sports Medicine, 2(8), 37–43. https://doi.org/10.1249/TJX.0000000000000032

Ekkekakis, P., & Tiller, N. B. (2022). Extraordinary Claims in the Literature on High-Intensity Interval Training: II. Are the Extraordinary Claims Supported by Extraordinary Evidence? Kinesiology Review, 1(aop), 1–14. https://doi.org/10.1123/kr.2022-0003

Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. ELife, 10, e71601. https://doi.org/10.7554/eLife.71601

Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PloS One, 4(5), e5738. https://doi.org/10.1371/journal.pone.0005738

Fanelli, D. (2010). “Positive” Results Increase Down the Hierarchy of the Sciences. PLOS ONE, 5(4), e10068. https://doi.org/10.1371/journal.pone.0010068

Fanelli, D., Costas, R., & Larivière, V. (2015). Misconduct Policies, Academic Culture and Career Stage, Not Gender or Pressures to Publish, Affect Scientific Integrity. PloS One, 10(6), e0127556. https://doi.org/10.1371/journal.pone.0127556

Fisher, R. A. (1926). The arrangement of field experiments. Journal of the Ministry of Agriculture, 33, 503–515. https://doi.org/10.23637/rothamsted.8v61q

Flanagin, A., Fontanarosa, P. B., & Bauchner, H. (2020). Preprints Involving Medical Research-Do the Benefits Outweigh the Challenges? JAMA, 324(18), 1840–1843. https://doi.org/10.1001/jama.2020.20674

Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505. https://doi.org/10.1126/science.1255484

Freemantle, N. (2001). Interpreting the results of secondary end points and subgroup analyses in clinical trials: Should we lock the crazy aunt in the attic? BMJ, 322(7292), 989–991. https://doi.org/10.1136/bmj.322.7292.989

Gandevia, S. (2021). Publications, replication and statistics in physiology plus two neglected curves. The Journal of Physiology, 599(6), 1719–1721. https://doi.org/10.1113/JP281360

Gardner, M. J., Altman, D. G., Jones, D. R., & Machin, D. (1983). Is the statistical assessment of papers submitted to the “British Medical Journal” effective? Br Med J (Clin Res Ed), 286(6376), 1485–1488. https://doi.org/10.1136/bmj.286.6376.1485

Garvican, L. A., Pottgiesser, T., Martin, D. T., Schumacher, Y. O., Barras, M., & Gore, C. J. (2011). The contribution of haemoglobin mass to increases in cycling performance induced by simulated LHTL. European Journal of Applied Physiology, 111(6), 1089–1101. https://doi.org/10.1007/s00421-010-1732-z

Gaspar, D. E. P., & Esteves, M. D. L. (2021). Awareness of the Misconduct in Sports Science Research. Annals of Applied Sport Science, 9(3), 0–0. https://doi.org/10.52547/aassjournal.934

Gasparyan, A. Y., Ayvazyan, L., Akazhanov, N. A., & Kitas, G. D. (2014). Self-correction in biomedical publications and the scientific impact. Croatian Medical Journal, 55(1), 61–72. https://doi.org/10.3325/cmj.2014.55.61

Gerrits, R. G., Jansen, T., Mulyanto, J., Berg, M. J. van den, Klazinga, N. S., & Kringos, D. S. (2019). Occurrence and nature of questionable research practices in the reporting of messages and conclusions in international scientific Health Services Research publications: A structured assessment of publications authored by researchers in the Netherlands. BMJ Open, 9(5), e027903. https://doi.org/10.1136/bmjopen-2018-027903

Gibbs, K. D., & Griffin, K. A. (2013). What Do I Want to Be with My PhD? The Roles of Personal Values and Structural Dynamics in Shaping the Career Interests of Recent Biomedical Science PhD Graduates. CBE Life Sciences Education, 12(4), 711–723. https://doi.org/10.1187/cbe.13-02-0021

Gibbs, N. M., & Gibbs, S. V. (2015). Misuse of “trend” to describe “almost significant” differences in anaesthesia research. British Journal of Anaesthesia, 115(3), 337–339. https://doi.org/10.1093/bja/aev149

Goodman, S. N. (1992). A comment on replication, P-values and evidence. Statistics in Medicine, 11(7), 875–879. https://doi.org/10.1002/sim.4780110705

Gopalakrishna, G., Ter Riet, G., Vink, G., Stoop, I., Wicherts, J. M., & Bouter, L. M. (2022). Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands. PloS One, 17(2), e0263023. https://doi.org/10.1371/journal.pone.0263023

Groenwold, R. H. H., & Dekkers, O. M. (2020). Measurement error in clinical research, yes it matters. European Journal of Endocrinology, 183(3), E3–E5. https://doi.org/10.1530/EJE-20-0550

Halperin, I., Vigotsky, A. D., Foster, C., & Pyne, D. B. (2018). Strengthening the Practice of Exercise and Sport-Science Research. International Journal of Sports Physiology and Performance, 13(2), 127–134. https://doi.org/10.1123/ijspp.2017-0322

Harris, J. D., Cvetanovich, G., Erickson, B. J., Abrams, G. D., Chahal, J., Gupta, A. K., McCormick, F. M., & Bach, B. R. (2014). Current status of evidence-based sports medicine. Arthroscopy: The Journal of Arthroscopic & Related Surgery: Official Publication of the Arthroscopy Association of North America and the International Arthroscopy Association, 30(3), 362–371. https://doi.org/10.1016/j.arthro.2013.11.015

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a

Higginson, A. D., & Munafò, M. R. (2016). Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions. PLOS Biology, 14(11), e2000995. https://doi.org/10.1371/journal.pbio.2000995

Hopkins, W. G., Marshall, S. W., Batterham, A. M., & Hanin, J. (2009). Progressive Statistics for Studies in Sports Medicine and Exercise Science. Medicine & Science in Sports & Exercise, 41(1), 3–12. https://doi.org/10.1249/MSS.0b013e31818cb278

How to spot dodgy academic journals. (2020). The Economist. https://www.economist.com/graphic-detail/2020/05/30/how-to-spot-dodgy-academic-journals

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124

Ioannidis, J. P. A., Boyack, K. W., & Baas, J. (2020). Updated science-wide author databases of standardized citation indicators. PLoS Biology, 18(10), e3000918. https://doi.org/10.1371/journal.pbio.3000918

Ioannidis, J. P. A., Ntzani, E. E., Trikalinos, T. A., & Contopoulos-Ioannidis, D. G. (2001). Replication validity of genetic association studies. Nature Genetics, 29(3), Article 3. https://doi.org/10.1038/ng749

Islam, H., & Gurd, B. J. (2020). Exercise response variability: Random error or true differences in exercise response? Experimental Physiology, 105(12), 2022–2024. https://doi.org/10.1113/EP089015

John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953

Jones, S. R., Carley, S., & Harrison, M. (2003). An introduction to power and sample size estimation. Emergency Medicine Journal, 20(5), 453–458. https://doi.org/10.1136/emj.20.5.453

Kardeş, S., Levack, W., Özkuk, K., Atmaca Aydın, E., & Seringeç Karabulut, S. (2020). Retractions in Rehabilitation and Sport Sciences Journals: A Systematic Review. Archives of Physical Medicine and Rehabilitation, 101(11), 1980–1990. https://doi.org/10.1016/j.apmr.2020.03.010

Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4

Khatra, O., Shadgan, A., Taunton, J., Pakravan, A., & Shadgan, B. (2021). A bibliometric analysis of the top cited articles in sports and exercise medicine. Orthopaedic Journal of Sports Medicine, 9(1), 2325967120969902. https://doi.org/10.1177/2325967120969902

Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biology, 14(5), e1002456. https://doi.org/10.1371/journal.pbio.1002456

King, E. G., Oransky, I., Sachs, T. E., Farber, A., Flynn, D. B., Abritis, A., Kalish, J. A., & Siracuse, J. J. (2018). Analysis of retracted articles in the surgical literature. American Journal of Surgery, 216(5), 851–855. https://doi.org/10.1016/j.amjsurg.2017.11.033

Kintisch, E. (2006). Poehlman Sentenced to 1 Year of Prison. Science.Org. https://www.science.org/content/article/poehlman-sentenced-1-year-prison

Kiss, A., Temesi, Á., Tompa, O., Lakner, Z., & Soós, S. (2021). Structure and trends of international sport nutrition research between 2000 and 2018: Bibliometric mapping of sport nutrition science. Journal of the International Society of Sports Nutrition, 18(1), 12. https://doi.org/10.1186/s12970-021-00409-5

Klimentidis, Y. C., Raichlen, D. A., Bea, J., Garcia, D. O., Wineinger, N. E., Mandarino, L. J., Alexander, G. E., Chen, Z., & Going, S. B. (2018). Genome-wide association study of habitual physical activity in over 377,000 UK Biobank participants identifies multiple variants including CADM2 and APOE. International Journal of Obesity (2005), 42(6), 1161–1176. https://doi.org/10.1038/s41366-018-0120-3

Knudson, D. (2009). Significant and meaningful effects in sports biomechanics research. Sports Biomechanics, 8(1), 96–104. https://doi.org/10.1080/14763140802629966

Knudson, D. (2012). Twenty-year trends of authorship and sampling in applied biomechanics research. Perceptual and Motor Skills, 114(1), 16–20. https://doi.org/10.2466/11.PMS.114.1.16-20

Knudson, D. (2017a). Twenty Years of Authorship, Sampling, and References in Kinesiology Research Reports. International Journal of Kinesiology in Higher Education, 1(2), 44–52. https://doi.org/10.1080/24711616.2017.1282760

Knudson, D. (2017b). Confidence crisis of results in biomechanics research. Sports Biomechanics, 16(4), 425–433. https://doi.org/10.1080/14763141.2016.1246603

Knudson, D. (2019). Judicious Use of Bibliometrics to Supplement Peer Evaluations of Research in Kinesiology in: Kinesiology Review Volume 8 Issue 2 (). Kinesiology Review, 8(2), 100–109. https://doi.org/doi.org/10.1123/kr.2017-0046

Kondro, W. (2005). Université de Montréal in the dark about fraud. CMAJ, 172(10), 1278-1278-a. https://doi.org/10.1503/cmaj.050455

Krans, B. (2022). Lawsuit: Coca-Cola Fake Ads About Obesity. Healthline.Com. https://www.healthline.com/health/coca-cola-false-advertising-unhealthy-drinks

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863

Landhuis, E. (2016). Scientific literature: Information overload. Nature, 535(7612), Article 7612. https://doi.org/10.1038/nj7612-457a

Lesser, L. I., Ebbeling, C. B., Goozner, M., Wypij, D., & Ludwig, D. S. (2007). Relationship between funding source and conclusion among nutrition-related scientific articles. PLoS Medicine, 4(1), e5. https://doi.org/10.1371/journal.pmed.0040005

Lohse, K. R., Sainani, K. L., Taylor, J. A., Butson, M. L., Knight, E. J., & Vickers, A. J. (2020). Systematic review of the use of “magnitude-based inference” in sports science and medicine. PLOS ONE, 15(6), e0235318. https://doi.org/10.1371/journal.pone.0235318

Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. Science (New York, N.Y.), 355(6325), 584–585. https://doi.org/10.1126/science.aal3618

Lucas, M. (2015). Conflicts of interest in nutritional sciences: The forgotten bias in meta-analysis. World Journal of Methodology, 5(4), 175–178. https://doi.org/10.5662/wjm.v5.i4.175

Maggio, L., Dong, T., Driessen, E., & Artino, A. (2019). Factors associated with scientific misconduct and questionable research practices in health professions education. Perspectives on Medical Education, 8(2), 74–82. https://doi.org/10.1007/s40037-019-0501-x

Mallapaty, S. (2020). China bans cash rewards for publishing papers. Nature, 579(7797), 18. https://doi.org/10.1038/d41586-020-00574-8

Marticorena, F. M., Carvalho, A., Oliveira, L. F. D., Dolan, E., Gualano, B., Swinton, P., & Saunders, B. (2021). Nonplacebo Controls to Determine the Magnitude of Ergogenic Interventions: A Systematic Review and Meta-analysis. Medicine and Science in Sports and Exercise, 53(8), 1766–1777. https://doi.org/10.1249/MSS.0000000000002635

Martin, J. J. (2020). Grants: The Good, the Bad, the Ugly, and the Puzzling. Kinesiology Review, 10(1), 18–28. https://doi.org/10.1123/kr.2020-0013

Mattioni Maturana, F., Martus, P., Zipfel, S., & NIEß, A. M. (2021). Effectiveness of HIIE versus MICT in Improving Cardiometabolic Risk Factors in Health and Disease: A Meta-analysis. Medicine and Science in Sports and Exercise, 53(3), 559–573. https://doi.org/10.1249/MSS.0000000000002506

Montero, D., & Lundby, C. (2017). Refuting the myth of non‐response to exercise training: ‘Non‐responders’ do respond to higher dose of training. The Journal of Physiology, 595(11), 3377–3387. https://doi.org/10.1113/JP273480

Moylan, E. C., & Kowalczuk, M. K. (2016). Why articles are retracted: A retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open, 6(11), e012047. https://doi.org/10.1136/bmjopen-2016-012047

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., du Sert, N. P., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021. https://doi.org/10.1038/s41562-016-0021

Murphy, J., Mesquida, C., Caldwell, A. R., Earp, B. D., & Warne, J. P. (2023). Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science. Sports Medicine (Auckland, N.Z.). https://doi.org/10.1007/s40279-022-01749-1

Nagele, P. (2003). Misuse of standard error of the mean (sem) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. BJA: British Journal of Anaesthesia, 90(4), 514–516. https://doi.org/10.1093/bja/aeg087

Nosek, B. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

Nosek, B. A., & Bar-Anan, Y. (2012). Scientific Utopia: I. Opening scientific communication. ArXiv:1205.1055 [Physics]. http://arxiv.org/abs/1205.1055

Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(6), 615–631. https://doi.org/10.1177/1745691612459058

Open Science Collaboration. (2015). PSYCHOLOGY. Estimating the reproducibility of psychological science. Science (New York, N.Y.), 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

Pabst, S., Brand, M., & Wolf, O. T. (2013). Stress effects on framed decisions: There are differences for gains and losses. Frontiers in Behavioral Neuroscience, 7, 142. https://doi.org/10.3389/fnbeh.2013.00142

Padilla, J., Leary, E., & Limberg, J. K. (2021). Identifying responders versus non-responders: Incorporation of controls is required for sound statistical inference. Experimental Physiology, 106(2), 375–376. https://doi.org/10.1113/EP089142

Prom-Wormley, E., Adkins, A., Waldman, I. D., & Dick, D. (2017). Critical Issues in Genetic Association Studies. In Psychological Science Under Scrutiny (pp. 221–249). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781119095910.ch12

Quan, W., Chen, B., & Shu, F. (2017). Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016). Aslib Journal of Information Management, 69(5), 486–502. https://doi.org/10.1108/AJIM-01-2017-0014

Robertson, P. S. (2020). Man & machine: Adaptive tools for the contemporary performance analyst. Journal of Sports Sciences, 38(18), 2118–2126. https://doi.org/10.1080/02640414.2020.1774143

Sabroe, I., Dockrell, D. H., Vogel, S. N., Renshaw, S. A., Whyte, M. K. B., & Dower, S. K. (2007). Identifying and hurdling obstacles to translational research. Nature Reviews. Immunology, 7(1), 77–82. https://doi.org/10.1038/nri1999

Sainani, K., & Chamari, K. (2022). Wish List for Improving the Quality of Statistics in Sport Science. International Journal of Sports Physiology and Performance, 17(5), 673–674. https://doi.org/10.1123/ijspp.2022-0023

Sainani, K. L., Borg, D. N., Caldwell, A. R., Butson, M. L., Tenan, M. S., Vickers, A. J., Vigotsky, A. D., Warmenhoven, J., Nguyen, R., Lohse, K. R., Knight, E. J., & Bargary, N. (2021). Call to increase statistical collaboration in sports science, sport and exercise medicine and sports physiotherapy. British Journal of Sports Medicine, 55(2), 118–122. https://doi.org/10.1136/bjsports-2020-102607

Sarabipour, S., Debat, H. J., Emmott, E., Burgess, S. J., Schwessinger, B., & Hensel, Z. (2019). On the value of preprints: An early career researcher perspective. PLoS Biology, 17(2), e3000151. https://doi.org/10.1371/journal.pbio.3000151

Scheel, A. M., Schijen, M., & Lakens, D. (2020). An excess of positive results: Comparing the standard Psychology literature with Registered Reports. PsyArXiv. https://doi.org/10.31234/osf.io/p6e9c

Schmid, S. L. (2017). Five years post-DORA: Promoting best practices for research assessment. Molecular Biology of the Cell, 28(22), 2941–2944. https://doi.org/10.1091/mbc.E17-08-0534

Schulz, K. F., Altman, D. G., Moher, D., & CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. BMC Medicine, 8, 18. https://doi.org/10.1186/1741-7015-8-18

Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. L. (2022). Reporting and transparent research practices in sports medicine and orthopaedic clinical trials: A meta-research study. BMJ Open, 12(8), e059347. https://doi.org/10.1136/bmjopen-2021-059347

Serodio, P., Ruskin, G., McKee, M., & Stuckler, D. (2020). Evaluating Coca-Cola’s attempts to influence public health ‘in their own words’: Analysis of Coca-Cola emails with public health academics leading the Global Energy Balance Network. Public Health Nutrition, 23, 1–7. https://doi.org/10.1017/S1368980020002098

Serra-Garcia, M., & Gneezy, U. (2021). Nonreplicable publications are cited more than replicable ones. Science Advances, 7(21), eabd1705. https://doi.org/10.1126/sciadv.abd1705

Shen, C., & Björk, B.-C. (2015). “Predatory” open access: A longitudinal study of article volumes and market characteristics. BMC Medicine, 13, 230. https://doi.org/10.1186/s12916-015-0469-2

Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., … Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/10.1177/2515245917747646

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Smaldino, P. E., & McElreath, R. (2016). The Natural Selection of Bad Science. Royal Society Open Science, 3(9), 160384. https://doi.org/10.1098/rsos.160384

Smoliga, J. M. (2020). Interpreting Biomarker Data After Concussion and Repeated Subconcussive Head Impacts: Challenges in Evaluating Brain Protection. JAMA Neurology, 77(12), 1477–1478. https://doi.org/10.1001/jamaneurol.2020.3467

Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021). Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour, 5(8), Article 8. https://doi.org/10.1038/s41562-021-01142-4

Sox, H. C., & Rennie, D. (2006). Research Misconduct, Retraction, and Cleansing the Medical Literature: Lessons from the Poehlman Case. Annals of Internal Medicine, 144(8), 609–613. https://doi.org/10.7326/0003-4819-144-8-200604180-00123

Spence, J. C., & Blanchard, C. (2001). Publication bias in sport and exercise psychology: The games we play. International Journal of Sport Psychology, 32(4), 386–399.

Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018). What meta-analyses reveal about the replicability of psychological research. Psychological Bulletin, 144(12), 1325–1346. https://doi.org/10.1037/bul0000169

Statista. (2020). Global sports nutrition & supplement market 2025. Statista. https://www.statista.com/statistics/450168/global-sports-nutrition-market/

Steen, R. G. (2011). Misinformation in the medical literature: What role do error and fraud play? Journal of Medical Ethics, 37(8), 498–503. https://doi.org/10.1136/jme.2010.041830

Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995). Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa. The American Statistician, 49(1), 108–112. https://doi.org/10.2307/2684823

Stuckler, D., Ruskin, G., & McKee, M. (2018). Complexity and conflicts of interest statements: A case-study of emails exchanged between Coca-Cola and the principal investigators of the International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE). Journal of Public Health Policy, 39(1), 49–56. https://doi.org/10.1057/s41271-017-0095-7

Teo, Y. Y. (2008). Common statistical issues in genome-wide association studies: A review on power, data quality control, genotype calling and population structure. Current Opinion in Lipidology, 19(2), 133–143. https://doi.org/10.1097/MOL.0b013e3282f5dd77

The Society for Transparency, Openness, and Replication in Kinesiology. (n.d.). Reports in Sport and Exercise. Retrieved March 3, 2022, from https://storkjournals.org/index.php/rrik

Thomas, J. R., Salazar, W., & Landers, D. M. (1991). What is missing in p less than .05? Effect size. Research Quarterly for Exercise and Sport, 62(3), 344–348. https://doi.org/10.1080/02701367.1991.10608733

Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26–30. https://doi.org/10.2307/1176337

Tijdink, J. K., Verbeke, R., & Smulders, Y. M. (2014). Publication Pressure and Scientific Misconduct in Medical Scientists. Journal of Empirical Research on Human Research Ethics, 9(5), 64–71. https://doi.org/10.1177/1556264614552421

Tiller, N. B. (2020). The Skeptic’s Guide to Sports Science: Confronting Myths of the Health and Fitness Industry. Routledge.

Tiller, N. B., Sullivan, J. P., & Ekkekakis, P. (2022). Baseless Claims and Pseudoscience in Health and Wellness: A Call to Action for the Sports, Exercise, and Nutrition-Science Community. Sports Medicine (Auckland, N.Z.). https://doi.org/10.1007/s40279-022-01702-2

Twomey, R., Yingling, V., Warne, J., Schneider, C., McCrum, C., Atkins, W., Murphy, J., Medina, C. R., Harlley, S., & Caldwell, A. (2021). The Nature of Our Literature: A Registered Report on the Positive Result Rate and Reporting Practices in Kinesiology. Communications in Kinesiology, 1(3), Article 3. https://doi.org/10.51224/cik.v1i3.43

Vagenas, G., Palaiothodorou, D., & Knudson, D. (2018). Thirty-year Trends of Study Design and Statistics in Applied Sports and Exercise Biomechanics Research. International Journal of Exercise Science, 11(1), 239–259.

van de Schoot, R., Winter, S. D., Griffioen, E., Grimmelikhuijsen, S., Arts, I., Veen, D., Grandfield, E. M., & Tummers, L. G. (2021). The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum. Frontiers in Psychology, 12, 621547. https://doi.org/10.3389/fpsyg.2021.621547

Van Noorden, R. (2010). Metrics: A profusion of measures. Nature, 465(7300), 864–866. https://doi.org/10.1038/465864a

Van Noorden, R., & Singh Chawla, D. (2019). Hundreds of extreme self-citing scientists revealed in new database. Nature, 572(7771), 578–579. https://doi.org/10.1038/d41586-019-02479-7

Vankov, I., Bowers, J., & Munafò, M. R. (2014). Article Commentary: On the Persistence of Low Power in Psychological Science. Quarterly Journal of Experimental Psychology, 67(5), 1037–1040. https://doi.org/10.1080/17470218.2014.885986

Viken, H., Reitlo, L. S., Zisko, N., Nauman, J., Aspvik, N. P., Ingebrigtsen, J. E., Wisløff, U., & Stensvold, D. (2019). Predictors of Dropout in Exercise Trials in Older Adults: The Generation 100 Study. Medicine & Science in Sports & Exercise, 51(1), 49–55. https://doi.org/10.1249/MSS.0000000000001742

Walters, J. (2015, August 11). Nutrition experts alarmed by nonprofit downplaying role of junk food in obesity. The Guardian. https://www.theguardian.com/society/2015/aug/11/obesity-junk-food-exercise-global-energy-balance-network-coca-cola

Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108

Watanabe, R. M. (2011). Statistical issues in gene association studies. Methods in Molecular Biology (Clifton, N.J.), 700, 17–36. https://doi.org/10.1007/978-1-61737-954-3_2

Wiedermann, C. J. (2016). Ethical publishing in intensive care medicine: A narrative review. World Journal of Critical Care Medicine, 5(3), 171–179. https://doi.org/10.5492/wjccm.v5.i3.171

Williams, C. J., Li, Z., Harvey, N., Lea, R. A., Gurd, B. J., Bonafiglia, J. T., Papadimitriou, I., Jacques, M., Croci, I., Stensvold, D., Wisloff, U., Taylor, J. L., Gajanand, T., Cox, E. R., Ramos, J. S., Fassett, R. G., Little, J. P., Francois, M. E., Hearon, C. M., … Coombes, J. S. (2021). Genome wide association study of response to interval and continuous exercise training: The Predict-HIIT study. Journal of Biomedical Science, 28(1), 37. https://doi.org/10.1186/s12929-021-00733-7

Wood, J., Freemantle, N., King, M., & Nazareth, I. (2014). Trap of trends to statistical significance: Likelihood of near significant P value becoming more significant with extra data. BMJ, 348, g2215. https://doi.org/10.1136/bmj.g2215

Yong, E. (2012). Nobel laureate challenges psychologists to clean up their act. Nature. https://doi.org/10.1038/nature.2012.11535

Downloads

Posted

2023-03-09