A Bayesian Approach to Interpret Intervention Effectiveness in Strength and Conditioning Part 2
Effect Size Selection and Application of Bayesian Updating
Keywords:S&C, Evaluation, Effect size, Bayesian, Effectiveness, Statistics
Background Effect sizes are commonly used to assess the effectiveness of interventions in strength and conditioning (S&C). The purposes of this large meta-analysis were to investigate the properties of two different effect size statistics and synthesize the large amount of data available in the form of informative Bayesian priors to quantify effectiveness of future S&C interventions.
Methods An online database and hand search of published and unpublished S&C intervention studies from the 1950’s onwards was conducted. Pre- and post-intervention data comprising means and standard deviations were extracted from outcomes categorized as: maximum strength, jump performance or sprint performance. Standardised mean difference (SMDpre) and percentage improvement (%Improve) obtained from the response ratio were calculated and modelled with 4-level Bayesian hierarchical meta-analysis models. Results were also used to create normally distributed priors which were incorporated into an accessible tool for assessing the effectiveness of future S&C interventions through the use of Bayesian updating.
Results Data from 628 studies comprising 5468 effect sizes were included in the analyses. Large differences were identified in the effect size distributions for maximum strength (pooled means: SMDpre =0.68 [95%CrI: 0.63 to 0.73]; %Improve = 14.3% [95%CrI: 13.3 to 15.4]) and sprint performance (pooled means: SMDpre =0.46 [95%CrI: 0.43 to 0.50]; %Improve = 6.8% [95%CrI: 6.3 to 7.3]). These differences were also reflected in development of Bayesian priors with the lowest means and largest relative variance obtained for sprint performance reflecting lower improvements in general, but also greater relative dispersion of results. Analysis of the tails of the effect size distributions indicated consistent overestimations of SMDpre values, likely caused by underestimated standard deviations.
Conclusions Future evaluations of S&C interventions are likely to be better performed and contextualised using Bayesian approaches featuring the information and informative priors developed in this meta-analysis. To facilitate an uptake of Bayesian methods within S&C, an easily accessible tool employing intuitive Bayesian updating was created. It is recommended that researchers and practitioners use the tool alongside the S&C specific threshold values, instead of continual isolated effect size calculations and Cohen’s generic values when evaluating the effectiveness of future S&C interventions. Researchers may choose to evaluate interventions using both SMDpre and percent improvement statistics given their different strengths and limitations.
Schoenfeld BJ, Ogborn DI and Krieger JW. Effect of repetition duration during resistance training on muscle hypertrophy: A systematic review. Sports Medicine. 2015;45(4):577-585. https://doi.org/10.1007/s40279-015-0304-0
Ralston GW, Kilgore L, Wyatt FB and Baker JS. The effects of weekly set volume on strength gain: A meta-analysis. Sports Medicine. 2017;47(12):2585-2601. https://doi.org/10.1007/s40279-017-0762-7.
Schoenfeld BJ, Ogborn DI and Krieger JW. Dose-response relationship between weekly resistance training volume and increases in muscle mass: A systematic review and meta- analysis. Journal of Sports Science. 2017;35(11):11073-1082. https://doi.org/10.1080/02640414.2016.1210197.
Williams TD, Tolusso DV, Fedewa MV and Esco MR. Comparison of periodized and non-periodized resistance training on maximal strength: A meta-analysis. Sports Medicine. 2017; 47(10):2083-2100. https://doi.org/10.1007/s40279-017-0734-y.
Gentil P, Arruda A, Souza D, Giessing J, Paoli A, Fisher J and Steele J. Is there any practical application of meta-analytical results in strength training? Frontiers in Physiology. 2017. https://doi.org/10.3389/fphys.2017.00001.
Rhea MR, Alvar BA, Burkett LN, Ball SD. A meta-analysis to determine the dose response for strength development. Medicine & Science in Sports and Exercise. 2003;35(3):456-64. https://doi.org/10.1249/01.MSS.0000053727.63505.D4.
Rhea MR. Determining the magnitude of treatment effects in strength training research through the use of the effect size. Journal of Strength & Conditioning Research. 2004;18:918-20. https://doi.org/10.1519/14403.1.
Swinton, PA. Burges, K. Hall, A. Greig L. Psyllas J. Aspe R. Maughan P. Murphy A. A Bayesian approach to interpret intervention effectiveness in strength and conditioning: Part 1. A meta-analysis to derive context-specific thresholds. Pre-print available from SportRχiv. https://doi.org10.51224/SRXIV.9.
Cohen, J. Statistical power analysis for the behavioral sciences. Second Edition. Hillsdale, NJ: Lawrence Erlbaum Associate. 1988.
Caldwell A, Vigotsky AD. A case against default effect sizes in sport and exercise science.
PeerJ 8:e10314. 2020. https://doi.org/10.7717/peerj.10314.
Dankel SJ and Loenneke JP. Effect sizes for paired data should use the change score variability rather than the pre-test variability. Journal of Strength and Conditioning Research. 2021;35(6):1773-1778. https://doi.org/10.1519/JSC.0000000000002946.
Baguley T. Standardized or simple effect size: what should be reported. British Journal of Psychology. 2009;100(3):603-617. https://doi.org/10.1348/000712608X377117.
Stone MH, Stone M and Sands WA. Principles and practice of resistance training.
Champaign IL: Human Kinetics. 2007.
Hedges LV, Gurevitch J and Curtis PS. The meta-analysis of response ratios in experimental ecology. Ecology. 1999;80(4):1150-1156. https://doi.org/10.2307/177062.
Friedrich, JO Adhikari NKJ and Beyene J. Ratio of means for analyzing continuous outcomes in meta-analysis performed as well as mean difference methods. Journal of Clinical Epidemiology. 2011;64(5):556–564. https://doi.org/10.1016/j.jclinepi.2010.09.016.
Koricheva J and Gurevitch J. Uses and misuses of meta-analysis in plant ecology. Journal of Ecology. 2014.102:828-844. https://doi.org/10.1111/1365-2745.12224.
Lajeunesse MJ. On the meta-analysis of response ratios for studies with correlated and multi-group designs. Ecology. 2011; 92(11):2049-2055. https://doi.org/10.2307/23034937.
Lajeunesse MJ. Bias and correction for the log response ratio in ecological meta-analysis. Ecology. 2015;96(8):2056-2063. https://doi.org/10.1890/14-2402.1.
Deb SK, Brown DR, Gough LA, Mclellan CP, Swinton PA, Sparks AS and Mcnaughton LR. Qing the effects of acute hypoxic exposure on exercise performance and capacity: A systematic review and meta-regression. European Journal of Sport Science. 2018;18(2):243-254. https://doi.org/10.1080/17461391.2017.1410233.
Hopkin WG, Marshall SW, Batterham AM and Hanin J. Progressive statistics for studies in sports medicine and exercise science. Medicine in Science and Sports and Exercise. 2009;41(1):3-13. https://doi.org/10.1249/MSS.0b013e31818cb278.
Weston M, Taylor KL, Batterham AM and Hopkins WG. Effects of low-volume high- intensity interval training (HIT) on fitness in adults: A meta-analysis of controlled and non-controlled trials. Sports Medicine. 2014;44(7):1005-1017. https://doi.org/10.1007/s40279-014-0180-z.
Vollard NBJ, Metcalfe RS and Williams S. Effect of number of sprints in an SIT session on change in VO2max: A meta-analysis. Medicine in Science and Sports and Exercise. 2017;49(6):1147-1156. https://doi.org/10.1249/MSS.0000000000001204.
Guizelini PC, de Aguiar RA, Denadai BS, Caputo F and Greco CC. Effect of resistance training on muscle strength and rate of force development in healthy older adults: A systematic review and meta-analysis. Experimental Gerontology. 2018;102:51-58. https://doi.org/10.1016/j.exger.2017.11.020.
Hespanhol L, Vallio CS, Costa LM and Saragiotto BT. Understanding and interpreting confidence and credible intervals around effect estimates. Brazilian Journal of Physical Therapy. 2019;23(4):290-301. https://doi.org/10.1016/j.bjpt.2018.12.006.
Goodman SN. Toward evidence-based medical statistics. 2: The Bayes factor. Annals of Internal Medicine. 1999;130(12):1005-1013. https://doi.org/10.7326/0003-4819-130-12- 199906150-00019.
Jones HE, Ades AE, Sutton AJ and Welton NJ. Use of a random effects meta-analysis in the design and analysis of a new clinical trial. Statistics in Medicine. 2018;37(30):4665- 4679. https://doi.org/10.1002/sim.7948.
Hox, JJ, Moerbeek M, Van de Schoot R. Multilevel Analysis. Techniques and applications.
rd edition. 2018. Routledge.
Gelman A. Prior distributions for variance parameters in hierarchical models. Bayesian Analysis. 2006;1(3):515-34.
Verardi V, Vermandele C. Univariate and multivariate outlier identification for skewed or heavy-tailed distributions. The Stata Journal. 2018;18(3):517-32. https://doi.org/10.1177/1536867X1801800303.
Bürkner PC. brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software. 2017;80(1):1-28. https://doi.org/ 10.18637/jss.v080.i01.
Gelman A, Carlin JB, Stern HS, Rubin DB. Bayesian Data Analysis: Taylor & Francis; 2014.
Pustejovsky JE. Using response ratios for meta-analysing single-case designs with behavioral outcomes. Journal of School Psychology. 2018;68:99-112. https://doi.org/10.1016/j.jsp.2018.02.003.
Mengersen KL, Drovandi CC, Robert CP, Pyne DB and Gore CJ. Bayesian estimation of small effects in exercise and sports science. Plos One. 2016. https://doi.org/10.1371/journal.pone.0147311.
Bernards JR, Sato K, Haff GG and Bazyler CD. Current research and statistical practices in sport science and a need for a change. Sports. 2017;5(4). https://doi.org/10.3390/sports5040087.
Turner AN, Brazier J, Bishop C, Chavda S, Cree J, and Read P. Data analysis for strength and conditioning coaches: Using Excel to analyze reliability, differences, and relationships. Strength and Conditioning Journal. 2015;37(1):76-83. https://doi.org/10.1519/SSC.0000000000000113.
Wasserstein RL, Schirm AL and Lazar NA. Moving to a world beyond “p<0.05”. The
American Statistician. 2019. https://doi.org/10.1080/00031305.2019.1583913.
Copyright (c) 2021 Paul Swinton, Katherine Burgess, Andy Hall, Leon Greig, John Psyllas, Rodrigo Aspe, Patrick Maughan, Andrew Murphy
This work is licensed under a Creative Commons Attribution 4.0 International License.