Preprint / Version 4

Capturing the "expert’s eye": Towards a better understanding and implementation of subjective performance evaluations in team sports


  • Johann Windt Vancouver Whitecaps Football Club
  • Keith Hamilton Simon Fraser University
  • Bruno D. Zumbo
  • David N. Cox
  • Ben Sporer Vancouver Whitecaps Football Club



Performance evaluation, Performance analysis, Validity theory, Talent identification


Subjective evaluations of athletic performance drive decision making across sporting organizations. Every day, based on their expertise and intuition, coaches select their starting lineups, scouts recommend or discourage teams from signing new potential players, and academy directors make decisions on which players move up or move out of a team’s academy system. While this intuitive evaluation of performance occurs constantly, little attention has been given to how this process can be formally designed, implemented, and assessed to capture these expert evaluations of performance more effectively and thereby better inform decision making within sports organizations.


Jokuschies N, Gut V, Conzelmann A. Systematizing coaches’ ‘eye for talent’: Player assessments based on expert coaches’ subjective talent criteria in top-level youth soccer. International Journal of Sports Science & Coaching 2017;12:565–76. doi:10.1177/1747954117727646

Williams AM, Reilly T. Talent identification and development in soccer. Journal of Sports Sciences 2000;18:657–67. doi:10.1080/02640410050120041

Atkinson G. Sport performance: variable or construct? Journal of Sports Sciences 2002;20:291–2. doi:10.1080/026404102753576053

Larkin P, O’Connor D. Talent identification and recruitment in youth soccer: Recruiter’s perceptions of the key attributes for player recruitment. PLOS ONE 2017;12:e0175716. doi:10.1371/journal.pone.0175716

Slimani M, Znazen H, Miarka B, et al. Maximum Oxygen Uptake of Male Soccer Players According to their Competitive Level, Playing Position and Age Group: Implication from a Network Meta-Analysis. J Hum Kinet 2019;66:233–45. doi:10.2478/hukin-2018-0060

Dugdale JH, Sanders D, Myers T, et al. A case study comparison of objective and subjective evaluation methods of physical qualities in youth soccer players. Journal of Sports Sciences 2020;38:1304–12. doi:10.1080/02640414.2020.1766177

Impellizzeri FM, Marcora SM. Test Validation in Sport Physiology: Lessons Learned from Clinimetrics. International Journal of Sports Physiology and Performance 2009;4:269–77. doi:10.1123/ijspp.4.2.269

Christensen MK. “An Eye for Talent”: Talent Identification and the “Practical Sense” of Top-Level Soccer Coaches. Sociology of Sport Journal 2009;26:365–82. doi:10.1123/ssj.26.3.365

Durlak JA, DuPre EP. Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. Am J Community Psychol 2008;41:327–50. doi:10.1007/s10464-008-9165-0

Saw AE, Main LC, Gastin PB. Monitoring Athletes Through Self-Report: Factors Influencing Implementation. J Sports Sci Med 2015;14:137–46.

Ward P, Windt J, Kempton T. Business Intelligence: How Sport Scientists Can Support Organisation Decision Making in Professional Sport. International Journal of Sports Physiology and Performance 2019;:1–10. doi:10.1123/ijspp.2018-0903

Decroos T, Bransen L, Van Haaren J, et al. Actions Speak Louder than Goals: Valuing Player Actions in Soccer. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage, AK, USA: : Association for Computing Machinery 2019. 1851–61. doi:10.1145/3292500.3330758

McIntosh S, Kovalchik S, Robertson S. Comparing subjective and objective evaluations of player performance in Australian Rules football. PLOS ONE 2019;14:e0220901. doi:10.1371/journal.pone.0220901

Bangsbo J, Iaia FM, Krustrup P. The Yo-Yo intermittent recovery test : a useful tool for evaluation of physical performance in intermittent sports. Sports Med 2008;38:37–51.

Buchheit M. The 30-15 intermittent fitness test: accuracy for individualizing interval training of young intermittent sport players. J Strength Cond Res 2008;22:365–74.

Spencer B, Jackson K, Bedin T, et al. Modelling the quality of player passing decisions in Australian Rules football relative to risk, reward and commitment. Front Psychol 2019;10. doi:10.3389/fpsyg.2019.01777

Fernández J, Barcelona FC, Bornn L, et al. Decomposing the Immeasurable Sport: A deep learning expected possession value framework for soccer. 2019;:20.

Bergkamp TLG, Frencken WGP, Niessen ASM, et al. How soccer scouts identify talented players. European Journal of Sport Science 2021;0:1–11. doi:10.1080/17461391.2021.1916081

Shaw L, Glickman M. Dynamic analysis of team strategy in professional football. ;:13.

Bransen L, Robberechts P, Van Haaren J, et al. Choke or Shine? Quantifying Soccer Players’ Abilities to Perform Under Mental Pressure. MIT Sloan Sports Analytics Conference 2019.

Saw AE, Main LC, Gastin PB. Monitoring the athlete training response: subjective self-reported measures trump commonly used objective measures: a systematic review. Br J Sports Med 2016;50:281–91. doi:10.1136/bjsports-2015-094758

Crowcroft S, Slattery K, McCleave E, et al. Do Athlete Monitoring Tools Improve a Coach’s Understanding of Performance Change? International Journal of Sports Physiology and Performance 2020;15:847–52. doi:10.1123/ijspp.2019-0338

Sieghartsleitner R, Zuber C, Zibung M, et al. Science or Coaches’ Eye? – Both! Beneficial Collaboration of Multidimensional Measurements and Coach Assessments for Efficient Talent Selection in Elite Youth Football. J Sports Sci Med 2019;18:32–43.

McCormack S, Jones B, Elliott D, et al. Coaches’ Assessment of Players Physical Performance: Subjective and Objective Measures are needed when Profiling Players. European Journal of Sport Science 2021;:1–11. doi:10.1080/17461391.2021.1956600

Cripps AJ, Hopper LS, Joyce C. Can coaches predict long-term career attainment outcomes in adolescent athletes? International Journal of Sports Science & Coaching 2019;14:324–8. doi:10.1177/1747954119848418

Montibeller G, von Winterfeldt D. Cognitive and Motivational Biases in Decision and Risk Analysis: Biases in Decision and Risk Analysis. Risk Analysis 2015;35:1230–51. doi:10.1111/risa.12360

Tversky A, Kahneman D. Judgment under Uncertainty: Heuristics and Biases. Science 1974;185:1124–31. doi:10.1126/science.185.4157.1124

Burton LJ, Mazerolle SM. Survey instrument validity part I: Principles of survey instrument development and validation in athletic training education research. Athletic Training Education Journal 2011;6:27–35.



2021-08-28 — Updated on 2021-09-06