Why humble farmers may in fact have bigger potatoes - A call for street-smart decision-making in sport
Rethinking decision-making in sport
Keywords:Decision Making, Heuristics, Forecasting, Bias, Variance, Prediction Modeling, Evidence, Expert
The main task of applied sport science is to inform decision-making in sports practice, that is, enabling practitioners to compare the expectable outcomes of different options (e.g. training programs). The “evidence” provided may range from group averages to multivariable prediction models. By contrast, many decisions are still largely based on the subjective, experience-based judgement of athletes and coaches. While for the research scientists this may seem “unscientific” and even “irrational”, it is important to realize the different perspectives: Science values novelty, universal validity, methodological rigor, and contributions towards long-term advancement. Practitioners are judged by the performance outcomes of contemporary, specific athletes. This makes out-of-sample predictive accuracy and robustness decisive requirements for useful decision support. At this point, researchers must concede that under the framework conditions of sport (small samples, multifactorial outcomes etc.) near certainty is unattainable, even with cutting-edge methods that might theoretically enable near-perfect accuracy. Rather, the sport ecosystem favors simpler rules, learning by experience, human judgement, and integration across different sources of knowledge. In other words, the focus of practitioners on experience and human judgement, complemented - but not superseded - by scientific evidence is probably street-smart after all. A major downside of this human-driven approach is the lack of science-grade evaluation and transparency. However, methods are available to merge the assets of data- and human-driven strategies and mitigate biases. This work presents the challenges of learning, forecasting and decision-making in sport as well as specific opportunities for turning the prevailing “evidence vs. eminence” contrast into a synergy.
Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning - Data mining, Inference, and Prediction. 2nd ed. Springer; 2017.
Hecksteden A, Schmartz G, Egyptien Y, Keller A, Meyer T. Forecasting soccer injuries by combining screening, monitoring and machine learning. Science and Medicine in Football. 2022. epub ahead of print. https://doi.org/10.1080/24733938.2022.2095006
Hecksteden A, Kellner R, Donath L. Dealing with small samples in football research. Science and Medicine in Football. Science and Medicine in Football. 2022;6:389-397. https://doi.org/10.1080/24733938.2021.1978106.
Greenland S., Mansournia M.A., Altman DG. Sparse data bias: a problem hiding in plain sight. BMJ. 2016; https://doi.org/10.1136/bmj,i1981
Porter T, Elnakouri A, Meyers EA, Shibayama T, Jayawickreme E, Grossmann I. Predictors and consequences of intellectual humility. Nat Rev Psychol. 2022;1:524-536. https://doi.org/10.1038/s44159-022-00081-9
Mellers B, Tetlock P, Arkes HR. Forecasting tournaments, epistemic humility and attitude depolarization. Cognition. 2019;188:19-26. https://doi.org/10.1016/j.cognition.2018.10.021
Senn S. Mastering variation: variance components and personalised medicine. Stat Med. 2016;35:966-977. https://doi.org/10.1002/sim.6739
Gigerenzer G, Brighton H. Homo Heuristicus: Why biased minds make better inferences. Topics in Cognitive Science. 2009;1:107-43. https://doi.org/10.1111/j.1756-8765.2008.01006.x.
Sieghartsleitner R, Zuber C, Zibung M, Conzelmann A. Science or Coaches' Eye? - Both! Beneficial Collaboration of Multidimensional Measurements and Coach Assessments for Efficient Talent Selection in Elite Youth Football. J Sports Sci Med. 2019;18:32-43.
Gigerenzer G, Gaissmaier W. Heuristic decision making. Annu Rev Psychol. 2011;62:451-82. https://doi.org/10.1146/annurev-psych-120709-145346
Ungar L, Mellers B, Satopää V, Baron J, Tetlock P, Ramos J, et al. The Good Judgment Project: A large scale test of different methods of combining expert predictions. Association for the Advancement of Artificial Intelligence. 2012; retrieved from: https://www.cis.upenn.edu/~ungar/papers/forecast_AAAI_MAGG.pdf.
Galton F. Vox populi. Nature. 1907;75:450-1.
Cureton KJ, Sparling PB, Evans BW, Johnson SM, Kong UD, Purvis JW. Effect of experimental alterations in excess weight on aerobic capacity and distance running performance. Med Sci Sports. 1978;10:194-9.
Zacharogiannis E, Paradisis G, Magos S, Plavoukos I, Dagli F, Pilianidis T, et al. The Effect Of Acute Body Mass Reduction On Metabolism And Endurance Running Performance. Med Sci Sports Exerc. 2017;49:194. https://doi.org/10.1249/01.mss.0000517367.65054.59
Savage LJ. The Foundations of Statistics,2nd ed. Dover Publications Inc.; 1972.
Reiter R. On closed world data bases. In: H. G, editor. Logic and Data Bases: Plenum Press, New York; 1978.
Hanheide M, Göbelbecker M, Horn GS, Pronobis A, Sjöö K, Aydemir A, et al. Robot task planning and exploration in open and uncertain worlds. Artificial Intelligence. 2017;247:119-50.
Lendrem DW, Lendrem BC, Woods D, Rowland-Jones R, Burke M, Chatfield M, et al. Lost in space: design of experiments and scientific exploration in a Hogarth Universe. Drug Discov Today. 2015;20:1365-1371. https://doi.org/10.1016/j.drudis.2015.09.015
Hogarth RM. Intuition: a challenge for psychological research and decision making. Psychol Inquiry. 2010;21:338-53.
Kellmann M, Bertollo M, Bosquet L, Brink M, Coutts AJ, Duffield R, et al. Recovery and Performance in Sport: Consensus Statement. Int J Sports Physiol Perform. 2018;19:1-6. https://doi.org/10.1123/ijspp.2017-0759
Taleb NN. Black Swans and the Domains of Statistics. The American Statistician. 2007;61:1-3. https://doi.org/10.1198/000313007X219996
Grove WM, Zald DH, Lebow BS, Snitz BE, Nelson C. Clinical versus mechanical prediction: a meta-analysis. Psychol Assess. 2000;12:19-30.
Mesquida C, Murphy J, Lakens D, Warne J. Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field. R Soc Open Sci. 2022;9:220946. https://doi.org/10.1098/rsos.220946
Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2:e124. https://doi.org/10.1371/journal.pmed.0020124
Ioannidis JP. Why replication has more scientific value than original discovery. Behav Brain Sci. 2018;41(E137). https://doi.org/10.1017/S0140525X18000729.
Dellerman D, Ebel P, Söllner M, Leimeister J. Hybrid Intelligence. Bus Inf Syst Eng. 2019;61:637-43.
Gibney E. Could machine learning fuel a reproducibility crisis in science? Nature. 2022;epub ahead of print. https://doi.org/10.1038/d41586-022-02035-w
Mellers B, Ungar L, Baron J, Ramos J, Gurcay B, Fincher K, et al. Psychological strategies for winning a geopolitical forecasting tournament. Psychol Sci. 2014 1;25:1106-1115. https://doi.org/10.1177/0956797614524255
Mellers BA, Tetlock PE. From discipline-centered rivalries to solution-centered science: Producing better probability estimates for policy makers. Am Psychol. 2019;74:290-300. https://doi.org/10.1037/amp0000429
Klein G, Calderwood R, Clinton-Cirocco A. Rapid decision making on the fire ground. Prodeedings of the Human Factors and Ergonomics Society Annual Meeting. 1986;30. https://doi.org/10.1177/154193128603000616
Wübben M, Wagenheim F. Instant customer base analysis: managerial heuristics often "get it right". J Mark. 2008;72:82-93.
Artinger F, Kozodoi N, Wangenheim F, Gigerenzer G. Recency: prediction with smart data. Am Mark Assoc Winter Conf Proc 2018;29:L2.
Barber B, Odean T. Trading Is Hazardous to Your Wealth: The Common Stock Investment Performance of Individual Investors. The Journal of Finance. 2000;55:773-806.
Kadlec D. Why US funds are not up to par. Time. 1997; January 27:32-33.
Snook B, Zito M, Bennell C, Taylor P. On the complexity and accuracy of geographic profiling strategies. J Quant Criminol. 2005;21:1-26.
Green LA, Mehr D. What alters physicians' decisions to admit to the coronary care unit? Journal of Family Practice. 1997;45:219-26.
Lichtman A. Predicting the next president: The keys to the White House. Rowman and Littlefield. 2016.
Serwe S, Frings C. Who will win Wimbledon? The recognition heuristic in predicting sports events. J Behav Decis Mak. 2006;19:321-32.
Gigerenzer G, Todd PM. Simple heuristics that make us smart: Oxford University Press; 1999.
Tversky A, Kahneman D. Judgment under Uncertainty: Heuristics and Biases. Science. 1974;185:1124-1131. https://doi.org/10.1126/science.185.4157.1124
Keller N, Katikopoulos K. On the role of psychological heuristics in operational research; and a demonstration in military stability operations. European Journal of Operational Research. 2015. https://doi.org/10.1016/j.ejor.2015.07.023
Keller N, Czienskowski U, Feufel M. Tying up loose ends: a method for constructing and evaluating decision aids that meet blunt and sharp-end goals. Ergonomics. 2014. https://doi.org/10.1080/00140139.2014.917204
Gigerenzer G, Reb J, Luan S. Smart Heuristics for Individuals, Teams, and Organizations. Annual Review of Organizational Psychology and Organizational Behavior. 2022;9:171-198. https://doi.org/10.1146/annurev-orgpsych-012420-090506
Tetlock PE, Mellers BA, Scoblic JP. Bringing probability judgments into policy debates via forecasting tournaments. Science. 2017;355:481-483. https://doi.org/10.1126/science.aal3147
Ulfelder J. Using the “Wisdom of (Expert) Crowds” To Forecast Mass Atrocities (Report). 2014. https://doi.org/10.2139%2Fssrn.2418980
Atanasov P, Witkowski J, Ungar L, Mellers B, Tetlock P. Small steps to accuracy: Incremental belief updaters are better forecasters. Organizational Behavior and Human Decision Processes. 2020;160:19-35. https://doi.org/10.1016/j.obhdp.2020.02.001
Bouchard C, Rankinen T. Individual differences in response to regular physical activity. Med Sci Sports Exerc. 2001;33(6 Suppl):S446-51; discussion S52-3.
Surowiecki J. The wisom of crowd: Why the many are smarter than the few and how collektive wisdom shapes business, economies, societies and nations. New York: Anchor; 2005.
https://goodjudgment.com/, Accessed 24.06.2022.
Kent DM, Steyerberg E, van Klaveren D. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. BMJ. 2018;10:k4245. https://doi.org/10.1136/bmj.k4245
Hecksteden A, Pitsch W, Julian R, Pfeiffer M, Kellmann M, Ferrauti A, et al. A New Method to Individualize Monitoring of Muscle Recovery in Athletes. Int J Sports Physiol Perform. 2017;12:1137-42. https://doi.org/10.1123/ijspp.2016-0120
Hecksteden A, Skorski S, Egger F, Buder F, Kellner R, Meyer T. Dwarfs on the shoulders of giants: Bayesian analysis with informative priors in elite sports research and decision making. Frontiers in Sports and Active Living. 2022. https://doi.org/10.3389/fspor.2022.793603.
Senn S. Transposed conditionals, shrinkage, and direct and indirect unbiasedness. Epidemiology. 2008;19:652-654; discussion 7-8. https://doi.org/10.1097/EDE.0b013e318181b3e3
Sottas PE, Baume N, Saudan C, Schweizer C, Kamber M, Saugy M. Bayesian detection of abnormal values in longitudinal biomarkers with an application to T/E ratio. Biostatistics. 2007;8:285-96. https://doi.org/10.1093/biostatistics/kxl009
Pearl J. Causal inference in statistics: An overview. Statistics Surveys. 2009;3:96-146. https://doi.org/10.1214/09-SS057
Munafo MR, Davey Smith G. Robust research needs many lines of evidence. Nature. 2018;553:399-401. https://doi.org/10.1038/d41586-018-01023-3
Copyright (c) 2023 Anne Hecksteden, Niklas Keller, Guangze Zhang, Tim Meyer, Thomas Hauser
This work is licensed under a Creative Commons Attribution 4.0 International License.