Reporting bias, not external focus
A robust Bayesian meta-analysis of the attentional focus literature
DOI:
https://doi.org/10.51224/SRXIV.304Keywords:
Skill acquisition, OPTIMAL theory, Metascience, Heterogeneity, Sport scienceAbstract
Evidence has ostensibly been accumulating over the past two decades suggesting that an external focus of attention is superior to an internal focus for the performance and learning of motor skills. Seven previous meta-studies have all reported evidence of external focus superiority---the most comprehensive of which concluded the benefits apply to motor skill (a) retention, (b) transfer, and (c) performance; results in (d) reduced electromyographic activity during performance, and that (e) more distal external foci are superior to proximal external foci for performance. Here, we re-analyzed these data using robust Bayesian meta-analysis methods that included several plausible models of publication bias. We found moderate to strong evidence of publication bias for all five analyses. After correcting for publication bias, estimated mean effects were negligible: g = .01 (performance), g = .15 (retention), g = .09 (transfer), g = .06 (electromyography), and g = -.01 (distance effect). Bayes factors indicated data favored the null for each analysis, ranging from BF01 = 1.3 (retention) to 5.74 (performance). Further, we found clear evidence of heterogeneity in each analysis, suggesting the impact of attentional focus depends on yet unknown contextual factors. Our results contradict the existing consensus that an external focus is always more effective than an internal focus. Instead, focus of attention appears to have a variety of effects that we cannot account for, and on average those effects are small to nil. These results parallel previous metascience suggesting publication bias has obfuscated the motor learning literature.
Metrics
References
Aust, F., & Barth, M. (2020). papaja: Prepare reproducible APA journal articles with R Markdown. https://github.com/crsh/papaja
Bacelar, M. F. B., Parma, J. O., Murrah, W. M., & Miller, M. W. (2022). Meta-analyzing enhanced expectancies on motor learning: Positive effects but methodological concerns. International Review of Sport and Exercise Psychology, 0(0), 1–30. https://doi.org/10.1080/1750984X.2022.2042839
Barth, M. (2022). tinylabels: Lightweight variable labels. https://cran.r-project.org/package=tinylabels
Bartoš, F., & Maier, M. (2020). RoBMA: An r package for robust bayesian meta-analyses. https://CRAN.R-project.org/package=RoBMA
Bartoš, F., Maier, M., Wagenmakers, E.-J., Doucouliagos, H., & Stanley, T. D. (2023). Robust Bayesian meta-analysis: Model-averaging across complementary publication bias
adjustment methods. Research Synthesis Methods, 14(1), 99–116. https://doi.org/10.1002/jrsm.1594
Braginsky, M., Mathur, M., & VanderWeele, T. J. (2023). PublicationBias: Sensitivity analysis for publication bias in meta-analyses. https://CRAN.R-project.org/package=PublicationBias
Chambers, C. (2019). What’s next for registered reports? Nature, 573(7773), 187–189.
Chang, W. (2023). Extrafont: Tools for using fonts. https://CRAN.R-project.org/package=extrafont
Chua, L.-K., Jimenez-Diaz, J., Lewthwaite, R., Kim, T., & Wulf, G. (2021). Superiority of external attentional focus for motor performance and learning: Systematic reviews and meta-analyses. Psychological Bulletin, 147, 618–645. https://doi.org/10.1037/bul0000335
Davids, K., Araújo, D., Shuttleworth, R., & Button, C. (2003). Acquiring skill in sport: A constraints-led perspective. International Journal of Computer Science in Sport.
DeBruine, L. (2023). Faux: Simulation for factorial designs. Zenodo. https://doi.org/10.5281/zenodo.2669586
Fitzpatrick, P., de Jonge, E., & Warnes, G. R. (2019). Daff: Diff, patch and merge for data.frames. https://CRAN.R-project.org/package=daff
Gagolewski, M. (2022). stringi: Fast and portable character string processing in R. Journal of Statistical Software, 103(2), 1–59. https://doi.org/10.18637/jss.v103.i02
Grgic, J., Mikulic, I., & Mikulic, P. (2021). Acute and Long-Term Effects of Attentional Focus Strategies on Muscular Strength: A Meta-Analysis. Sports, 9(11, 11), 153. https://doi.org/10.3390/sports9110153
Grgic, J., & Mikulic, P. (2022). Effects of Attentional Focus on Muscular Endurance: A Meta-Analysis. International Journal of Environmental Research and Public Health,19, 89. https://doi.org/10.3390/ijerph19010089
Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The Theory of Event Coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24(5), 849–878. https://doi.org/10.1017/S0140525X01000103
Hussien, J., Gignac, L., Shearer, L., & Ste-Marie, D. M. (2023a). The path to translating focus of attention research into Canadian physiotherapy, Part 2: Physiotherapist interviews reveal impacting factors and barriers to focus of attention use. Journal of Motor Learning and Development, 1(aop), 1–21. https://doi.org/10.1123/jmld.2022-0053
Hussien, J., Gignac, L., Shearer, L., & Ste-Marie, D. M. (2023b). The path to translating focus of attention research into Canadian physiotherapy, Part 3: Designing a workshop
through consultation with physiotherapists and focus of attention researchers. Journal of Motor Learning and Development, 1(aop), 1–16. https://doi.org/10.1123/jmld.2022-0067
Hussien, J., & Ste-Marie, D. M. (2023). The path to translating focus of attention research into Canadian physiotherapy, Part 1: Physiotherapists’ self-reported focus of attention
use via a study-specific questionnaire. Journal of Motor Learning and Development, 11(1), 86–99. https://doi.org/10.1123/jmld.2022-0052
Iannone, R., Cheng, J., Schloerke, B., Hughes, E., Lauer, A., & Seo, J. (2023). Gt: Easily create presentation-ready display tables. https://CRAN.R-project.org/package=gt
Johnson, L., Burridge, J., Ewings, S., Westcott, E., Gayton, M., & Demain, S. (2023). Principles into Practice: An Observational Study of Physiotherapists use of Motor Learning Principles in Stroke Rehabilitation. Physiotherapy, 118, 20–30. https://doi.org/10.101 6/j.physio.2022.06.002
Kay, M. (2023). ggdist: Visualizations of distributions and uncertainty. https://doi.org/10.5281/zenodo.3879620
Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217.
Kim, T., Jimenez-Diaz, J., & Chen, J. (2017). The effect of attentional focus in balancing tasks: A systematic review with meta-analysis. Journal of Human Sport and Exercise, 12, 463–479. https://doi.org/10.14198/jhse.2017.122.22
Kompf, J. (2015, June 8). Research Shows the Best Way to Cue a Client. The PTDC. https://www.theptdc.com/articles/why-external-focus-of-attention-maximizes-motor-performance-and-learning
Kuzdub, M. (2022, April 14). External vs Internal Cues: Does it Make a Difference? Mattspoint Tennis. https://www.mattspoint.com/blog/difference-external-vs-internal-cues
Lee, T. D., & Carnahan, H. (2021). Motor Learning: Reflections on the Past 40 Years of Research. Kinesiology Review, 10(3), 274–282. https://doi.org/10.1123/kr.2021-0018
Li, D., Zhang, L., Yue, X., Memmert, D., & Zhang, Y. (2022). Effect of Attentional Focus on Sprint Performance: A Meta-Analysis. International Journal of Environmental Research and Public Health, 19, 6254. https://doi.org/10.3390/ijerph19106254
Lo, A. (2019, January 16). Are Your Cues Holding Your Clients Back? The Physio Detective. https://physiodetective.com/are-your-cues-holding-your-clients-back/
Lohse, K., Buchanan, T., & Miller, M. (2016). Underpowered and Overworked: Problems With Data Analysis in Motor Learning Studies. Journal of Motor Learning and Development, 4(1), 37–58. https://doi.org/10.1123/jmld.2015-0010
Magne, G., & Edge, R. (2017, December 7). Language of Movement. Precision Movement & Therapies- Downtown Winnipeg Physio, Chiropractic, Massage & Athletic Therapy Clinic. https://precisionmovement.ca/language-of-movement/
Makaruk, H., Starzak, M., & Porter, J. M. (2020). Influence of Attentional Manipulation on Jumping Performance: A Systematic Review and Meta-Analysis. Journal of Human Kinetics, 75(1), 65–75. https://doi.org/10.2478/hukin-2020-0037
McKay, B., Bacelar, M. F. B., Parma, J. O., Miller, M. W., & Carter, M. J. (2023). The combination of reporting bias and underpowered study designs has substantially exaggerated the motor learning benefits of self-controlled practice and enhanced expectancies: A meta-analysis. International Review of Sport and Exercise Psychology, 1–21. https://doi.org/10.1080/1750984X.2023.2207255
McKay, B., Hussien, J., Vinh, M.-A., Mir-Orefice, A., Brooks, H., & Ste-Marie, D. M. (2022). Meta-analysis of the reduced relative feedback frequency effect on motor learning and performance. Psychology of Sport and Exercise, 61, 102165. https://doi.org/10.1016/j.psychsport.2022.102165
McKay, B., Yantha, Z., Hussien, J., Carter, M., & Ste-Marie, D. (2022). Meta-Analytic Findings of the Self-Controlled Motor Learning Literature: Underpowered, Biased, and Lacking Evidential Value. Meta-Psychology, 6. https://doi.org/10.15626/MP.2021.2803
Mesquida, C., Murphy, J., Lakens, D., & Warne, J. (2022). Replication concerns in sports and exercise science: A narrative review of selected methodological issues in the field. Royal Society Open Science, 9(12), 220946. https://doi.org/10.1098/rsos.220946
Neumann, T. (2017). Say the Magic Words: Internal vs External Coaching Cues. MyTPI. https://www.mytpi.com/articles/swing/say_the_magic_words_internal_vs_external_coaching_cues
Nicklas, A., Rein, R., Noël, B., & Klatt, S. (2022). A meta-analysis on immediate effects of attentional focus on motor tasks performance. International Review of Sport and Exercise Psychology, 1–36. https://doi.org/10.1080/1750984X.2022.2062678
Ooms, J. (2023). Magick: Advanced graphics and image-processing in r. https://CRAN.R-project.org/package=magick
Otte, W. M., Vinkers, C. H., Habets, P. C., IJzendoorn, D. G. P. van, & Tijdink, J. K. (2022). Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance. PLOS Biology, 20(2), e3001562. https://doi.org/10.1371/journal.pbio.3001562
Pedersen, T. L. (2022). Patchwork: The composer of plots. https://CRAN.R-project.org/package=patchwork
Peterson, A. (2019, October 22). Internal vs. External Focus for Baseball and Softball Hitters. The Hitting Vault. https://thehittingvault.com/internal-external-focus/
Prinz, W. (1990). A Common Coding Approach to Perception and Action. In O. Neumann & W. Prinz (Eds.), Relationships Between Perception and Action: Current Approaches (pp. 167–201). Springer. https://doi.org/10.1007/978-3-642-75348-0_7
R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
Re, A. C. D. (2013). Compute.es: Compute effect sizes. In R Package. https://cran.r-project.org/package=compute.es
Rohatgi, A. (2022). Webplotdigitizer: Version 4.6. https://automeris.io/WebPlotDigitizer
Scheel, A. M., Schijen, M. R., & Lakens, D. (2021). An excess of positive results: Comparing the standard psychology literature with registered reports. Advances in Methods and Practices in Psychological Science, 4(2), 25152459211007467.
Sievert, C. (2020). Interactive web-based data visualization with r, plotly, and shiny. Chapman; Hall/CRC. https://plotly-r.com
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
Smale, K. (2021, February 14). Coaching Cues: External vs Internal Focus of Attention. Apex Skating. https://apexskating.com/blogs/blog/coaching-cues-external-vs-internal-focus-of-attention
Ste-Marie, D. M., Carter, M. J., & Yantha, Z. D. (2020). Self-controlled learning: Current findings, theoretical perspectives, and future directions. In N. J. Hodges & A.M. Williams (Eds.), Skill acquisition in sport: Research, theory, and practice (3rd ed.). Routledge. https://doi.org/10.4324/9781351189750-7
Twomey, R., Yingling, V., Warne, J., Schneider, C., McCrum, C., Atkins, W., Murphy, J., Medina, C. R., Harlley, S., & Caldwell, A. (2021). The Nature of Our Literature: A Registered Report on the Positive Result Rate and Reporting Practices in Kinesiology. Communications in Kinesiology, 1(3, 3). https://doi.org/10.51224/cik.v1i3.43
Ushey, K. (2023). Renv: Project environments. https://CRAN.R-project.org/package=renv
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03
Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. Springer-Verlag New York. https://ggplot2.tidyverse.org
Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L. D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T. L., Miller, E., Bache, S. M., Müller, K., Ooms, J., Robinson, D., Seidel, D. P., Spinu, V., … Yutani, H. (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686
Winkelman, N. (2015). Sample External Cues. IDEA Health & Fitness Association. https://www.ideafit.com/personal-training/sample-external-cues/
Wulf, G. (2013). Attentional focus and motor learning: A review of 15 years. International Review of Sport and Exercise Psychology, 6(1), 77–104. https://doi.org/10.1080/1750984X.2012.723728
Wulf, G. (2007). Attention and motor skill learning. Human Kinetics.
Wulf, G., Höß, M., & Prinz, W. (1998). Instructions for Motor Learning: Differential Effects of Internal Versus External Focus of Attention. Journal of Motor Behavior, 30(2), 169–179. https://doi.org/10.1080/00222899809601334
Wulf, G., & Lewthwaite, R. (2016). Optimizing performance through intrinsic motivation and attention for learning: The OPTIMAL theory of motor learning. Psychonomic Bulletin & Review, 23(5), 1382–1414. https://doi.org/10.3758/s13423-015-0999-9
Wulf, G., & Prinz, W. (2001). Directing attention to movement effects enhances learning: A review. Psychonomic Bulletin & Review, 8(4), 648–660. https://doi.org/10.3758/BF03196201
Zhu, H. (2021). kableExtra: Construct complex table with ’kable’ and pipe syntax. https://CRAN.R-project.org/package=kableExtra
Downloads
Posted
Versions
- 2024-04-23 (3)
- 2023-06-20 (2)
- 2023-06-16 (1)
License
Copyright (c) 2023 Brad McKay, Abbey E. Corson, Jeswende Seedu, Celeste S. De Faveri, Hibaa Hasan, Kristen Arnold, Faith C. Adams, Michael J. Carter (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
